Development issueshttps://lab.civicrm.org/groups/dev/-/issues2023-10-13T05:11:14Zhttps://lab.civicrm.org/dev/core/-/issues/2945Make CRM_Core_Smarty::singleton() use Civi::statics instead of a static var2023-10-13T05:11:14ZDaveDMake CRM_Core_Smarty::singleton() use Civi::statics instead of a static varDuring unit tests, static vars can be problematic because they don't get reset between tests. There's a point within smarty where if a test fails at that point then its secure mode doesn't get reset. It's come up a couple times and then ...During unit tests, static vars can be problematic because they don't get reset between tests. There's a point within smarty where if a test fails at that point then its secure mode doesn't get reset. It's come up a couple times and then all the tests that use smarty on certain forms fail after it. It's not easy to track down if you haven't seen it before.
There's an argument civi code should be smarty secure-mode-compliant instead, but it isn't at the moment.https://lab.civicrm.org/dev/core/-/issues/2940Remove requirement to have a bounce email account setup2022-09-30T20:45:40ZseamusleeRemove requirement to have a bounce email account setupWith some more modern SMTP providers bounce email accounts may not be necessary for CiviMail Bounces to work because e.g. with Amazon SES / Sendgrid you can get the bounces processed via webhooks instead.
However within the code base we...With some more modern SMTP providers bounce email accounts may not be necessary for CiviMail Bounces to work because e.g. with Amazon SES / Sendgrid you can get the bounces processed via webhooks instead.
However within the code base we assume we need a bounce account to generate stuff like [no-reply email address](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/Domain.php#L364) or when we are getting [message ids](https://github.com/civicrm/civicrm-core/blob/513f6a469927c5dc5a41905feb7c3cff2cf15f0b/CRM/Campaign/BAO/Petition.php#L548)
It would be nice to get rid of this reliance on it being the default mail account which is the bounce processing one https://github.com/civicrm/civicrm-core/blob/513f6a469927c5dc5a41905feb7c3cff2cf15f0b/CRM/Core/BAO/MailSettings.php#L59
cc @JoeMurrayMonish DebMonish Debhttps://lab.civicrm.org/dev/core/-/issues/2935Print/Merge produces blank results when using a docx or odt template file2023-07-27T10:57:01ZChrisHardiePrint/Merge produces blank results when using a docx or odt template fileOverview
----------------------------------------
When trying to generate thank you letters or other printed/merged documents, and when uploading a .docx or .odt template file to use, CiviCRM generates blank output (e.g. a blank PDF file...Overview
----------------------------------------
When trying to generate thank you letters or other printed/merged documents, and when uploading a .docx or .odt template file to use, CiviCRM generates blank output (e.g. a blank PDF file, a blank .docx file). When generating the same document without uploading a template file, CiviCRM successfully generates output that contains the desired document body.
Reproduction steps
----------------------------------------
1. Find one or more contacts, or contributions, or other entities that can be printed/merged
1. Select an action that involves print/merge (e.g. thank you letters)
1. Upload a template document to use for the printed/merged output, in .odt or .docx format
1. Generate the merged document in any format (pdf, docx, html, odt)
1. Observe that the resulting document is blank
Expected behaviour
----------------------------------------
When a merged/generated document is produced, it should use the uploaded template and the specified document body, and not be blank.
Environment information
----------------------------------------
* __Browser:__ Attempted with Safari 15.1 and Chrome 96.0.4664.27 on macOS
* __CiviCRM:__ 5.42.0
* __PHP:__ 7.4
* __CMS:__ WordPress 5.8.1https://lab.civicrm.org/dev/civicrm-asset-plugin/-/issues/19Issues with the extension name vs key while building2021-11-09T12:08:26ZVangelisPIssues with the extension name vs key while buildingI've found an issue that relates with the way this component is reading the information from the extension and creates the relevant folder.
One example is `stripe` (but many more are having the same issue). Here is how we include it int...I've found an issue that relates with the way this component is reading the information from the extension and creates the relevant folder.
One example is `stripe` (but many more are having the same issue). Here is how we include it into composer:
```json
{
"name": "civicrm/extensions",
"description": "CiviCRM extensions",
"type": "civicrm-extension",
"require": {
"civicrm/stripe": "6.6.3"
},
"repositories": [
{
"type": "vcs",
"url": "https://lab.civicrm.org/extensions/stripe.git"
}
]
}
```
Now, due to [this line](https://lab.civicrm.org/dev/civicrm-asset-plugin/-/blob/master/src/ExtensionAssetRule.php#L17) , what is actually being created under the folder `web/libraries/civicrm` is `com.drastikbydesign.stripe` which is the `key` of the extension and not the project name (shortname).
In the meantime, stripe is being downloaded under the vendor folder as `stripe` and not as `com.drastikbydesign.stripe`.
If stripe is including some JS/CSS, those will NOT be read in CiviCRM as Civi is searching for the folder `web/libraries/civicrm/stripe` and not `web/libraries/civicrm/com.drastikbydesign.stripe`.
I can think of 2 possible solutions here:
1. instead of the attribute `key`, use (?) the `<file>` tag?
2. keep reading the attribute `key` as-is but also pass the actual folder name (which should be under the attribute `file`)
Any thoughts?https://lab.civicrm.org/dev/core/-/issues/2929Geocoding failures kill contributions2023-10-23T16:08:42ZandrewcormickdockeryGeocoding failures kill contributionsOverview
----------------------------------------
Last night between about 6pm and midnight AEDT, it appears that Nominatim (the Open Street Map API provider) had an intermittent outage. Some requests were receiving a 502 error. We hav...Overview
----------------------------------------
Last night between about 6pm and midnight AEDT, it appears that Nominatim (the Open Street Map API provider) had an intermittent outage. Some requests were receiving a 502 error. We have configured this as our Geocoding provider in CiviCRM under /civicrm/admin/setting/mapping?reset=1
People who were using CiviCRM in ways which caused the Geomapping provider to be called were receiving fatal errors as a result. This included people who were making donations (i.e. entering their address on the form which in turn was handed to the Geomapping provider for geocoding).
Current behaviour
----------------------------------------
When a request to the Geomapping provider fails (for example with a 502 error), the generating transaction within CiviCRM returns a fatal error and does not complete. This includes financial transactions, and indeed any administrative function that involves adding or updating a contact's address.
Expected behaviour
----------------------------------------
A Geomapping provider failure should never cause financial transactions to fail. I can't think of any reason why adding or updating an address should fail either. Fatal errors should only occur when an issue occurs which genuinely prevent a transaction from succeeding. A payment can still be processed regardless of whether that person's address can be geocoded or not. I expect a 502 Bad Gateway error from Nominatim would be logged, but never cause an address to fail to be added/updated. Obviously that would occur without the latitude/longitude being recorded. I can't even see the utility in informing the user. It's a system admin problem.
Environment information
----------------------------------------
* __CiviCRM:__ _5.35.2_
* __PHP:__ _7.3__
* __CMS:__ _Drupal 7.82_
* __Database:__ _MariaDB 10.4.21_
* __Web Server:__ _Nginx 1.10.3_
Comments
----------------------------------------
_Anything else you would like the reviewer to note._https://lab.civicrm.org/dev/core/-/issues/2928Expand Gender options?2024-01-16T16:07:07ZJoeMurrayExpand Gender options?Overview
----------------------------------------
Currently core has a Gender field with options in tarball of: Male, Female, Other. There has been a great deal of social change in this area with widespread use of additional terms. In ad...Overview
----------------------------------------
Currently core has a Gender field with options in tarball of: Male, Female, Other. There has been a great deal of social change in this area with widespread use of additional terms. In addition, it might be useful to have default questions and options for whether a contact is transgendered and what pronouns they would like.
Example use-case
----------------------------------------
1. Click on **Contacts -> New Individual**.
1. Enter **First Name** and **Last Name** and click **Save**.
Current behaviour
----------------------------------------
![2021-10-26_11-46-51](/uploads/f5985494ccec6ce0c0c143cc688014b2/2021-10-26_11-46-51.png)
Proposed behaviour
----------------------------------------
I'm going to break up the proposal into several parts that can be discussed and possibly approved separately.
1. Add to the list of gender options after Other: Prefer not to specify.
2. After Male and before Other, insert new gender options: Non-binary, Genderfluid.
2. Change from Female to Female/woman and from Male to Male/man.
3. Change the Gender field from single select to multiselect.
4. Add a new core field under Gender: Are you transgender? Yes, No, Prefer not to specify.
5. Add a new core field under the above: What pronouns do you use? He/him/his, She/her/hers, They/them/theirs, Zie/hir/hirs, Other.
6. Make the field for Pronouns multiselect.
An alternative that might make sense is to do some of this in an extension. Another alternative would be to assist with best practices by providing questions or options but defaulting to disabled.
Comments
----------------------------------------
Here is a note from an organization that works in this area in response to my query on behalf of a client for a longer list of genders:
> Coming up with a universal list of terminology to describe trans people can be difficult, since the exact terms used to describe the concepts involved varies dramatically from language to language and culture to culture. While trans communities in Canada and the US tend to use the same terminology, it can vary even with other majority Anglophone countries, to say nothing of the rest of the world.
>
> There is a lot of discussion out there in the data world about how best to record sexual orientation and gender identity (or “SOGI”) data, and best practices depend on the field in question. That said, a good starting point is to divide things into four questions framed like so:
>
> 1. What is your gender? (select all that apply)
> a. Female/woman
> b. Male/man
> c. Non-binary
> d. Genderfluid
> e. Other
> f. Prefer not to specify
>
> 2. Are you transgender?
> a. Yes
> b. No
> c. Prefer not to specify
>
> 3. What pronouns do you use? (select all that apply)
> a. He/him/his
> b. She/her/hers
> c. They/them/theirs
> d. Zie/hir/hirs
> e. Other
>
> Breaking things out like this has several advantages. First, it recognizes that “transgender” is not itself a gender identity, but is an umbrella category that includes certain men, women, and non-binary and genderfluid people. It simplifies the first question, keeping a shorter list for people to have to scroll through to find the proper term to describe themselves. It also allows organizations to easily modify the list to fit their specific use cases – an organization working in Samoa or with a significant Samoan population could add “fa’afafine” to the list while one dealing with a significant Native/First Nations population could add “two-spirit.”
>
> It also permits trans individuals to interact with the organization without either lying or outing themselves, by not forcing them to select either (for example) “cisgender woman” or “transgender woman.” (If you’re not familiar with the term, ‘cisgender’ is a word indicating a person who is not ‘transgender’.) In fact, it also allows the organization to sort their data in such a way that they don’t unnecessarily out trans people to staff and volunteers, by allowing them to pull a list of all women or all men that doesn’t automatically flag whether or not they are transgender.
>
> In addition, it reflects the fact that not all non-binary and genderfluid people consider themselves to fit under the “transgender” umbrella – this is a complicated issue that’s largely unknown outside of the larger LGBTQ community, and is often disregarded by organizations working with these communities.
>
> Finally, it flags the most useful information – what pronouns to use when referring to the person – into a separate easy to reference field that can be included on a display screen or other record without being specifically tied to someone’s transgender status.
>
> Of course, this is only scratching the surface of the issues that might be relevant to a specific organization’s needs. For instance, they should also consider having a separate “legal name” and “preferred name” field – the former is something that would only be referenced when writing checks or otherwise transacting business in the person’s legal name, but have the “preferred name” be what actually displays for all other interactions with the person. This would also allow cisgender people who primarily go by a short form, initials, or nickname to have that be displayed instead of their full name.
>
> A medical organization, meanwhile, might need to collect information on whether or not someone is intersex, or what their “gender assigned at birth” is, and there can be a lot of ways to capture this information based on how the organization interacts with its staff and patients.
>
> For additional discussion on collecting SOGI data, with multiple examples, please take a look at these resources:
>
> https://www.thetrevorproject.org/wp-content/uploads/2021/07/Measuring-Youth-Sexual-Orientation-and-Gender-Identity.pdf
>
> https://williamsinstitute.law.ucla.edu/quick-facts/survey-measures/JoeMurrayJoeMurrayhttps://lab.civicrm.org/dev/financial/-/issues/189Creating an Order fails with `Line item total doesn't match total amount` due...2023-06-19T09:18:26ZhaystackCreating an Order fails with `Line item total doesn't match total amount` due to the mismatch of 2 vs 6 decimal placesOpening this instead of conflating two issues in #188 - specifically referring to [this thread](https://lab.civicrm.org/dev/financial/-/issues/188#note_66286). As the subject says, creating an Order can fail with the error `Line item tot...Opening this instead of conflating two issues in #188 - specifically referring to [this thread](https://lab.civicrm.org/dev/financial/-/issues/188#note_66286). As the subject says, creating an Order can fail with the error `Line item total doesn't match total amount` because of the mismatch of 2 vs 6 decimal places when calculating the Order total from the amounts and tax amounts in the Line Items.
Say, for example, I have two Taxable Line Items in an Order both of which have had their Tax Amount pre-calculated - in this case by WooCommerce but I assume this could equally be the result from any external Payment Processor. All the amounts in the following params are correct as far as WooCommerce is concerned. Both CiviCRM and WooCommerce have `19.37910000` configured as the relevant Tax Rate.
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 5
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2250
[invoice_id] => 2250_woocommerce
[receive_date] => 2021-10-13 19:34:30
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 77.59 <-- The pre-calculated Total Amount
[tax_amount] => 12.59 <-- The pre-calculated Total Tax Amount
[source] => Shop
[campaign_id] => 3
[note] => Solstice Ticket (Bass) x 1, Solstice Ticket (Tenor) x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[price_set_id] => 8
[fee_level] => Bass
[fee_amount] => 29.84
[source] => Shop: Solstice Ticket (Bass)
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 9
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00 <-- The Line Total
[tax_amount] => 4.84 <-- The pre-calculated Tax Amount
[label] => Bass
[entity_table] => civicrm_participant
[financial_type_id] => 5
[price_field_value_id] => 19
)
)
)
[18] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[price_set_id] => 8
[fee_level] => Tenor
[fee_amount] => 47.75
[source] => Shop: Solstice Ticket (Tenor)
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 9
[unit_price] => 40.00
[qty] => 1
[line_total] => 40.00 <-- The Line Total
[tax_amount] => 7.75 <-- The pre-calculated Tax Amount
[label] => Tenor
[entity_table] => civicrm_participant
[financial_type_id] => 5
[price_field_value_id] => 20
)
)
)
)
)
```
The Order API recalculates `$order->getTotalAmount()` and [then compares it with what's passed in via the API](https://github.com/civicrm/civicrm-core/blob/f73d3ac9a9979d8a616b0e2df53a459f7e6a84d7/api/v3/Order.php#L113). However, the rounding to 6 decimal places (rather than 2) triggers `CRM_Contribute_Exception_CheckLineItemsException` because `$params['total_amount'] = 77.59` does not equal `$order->getTotalAmount() = 77.596415`.
WooCommerce appears to _round_ the Line Items before _summing_ them, so:
* Line Item 1: `1.193791 * 25 = 29.844775 => 29.84`
* Line Item 2: `1.193791 * 40 = 47.75164 => 47.75`
* Total of rounded Line Items: `77.59`
CiviCRM, appears to _sum_ the Line Items before _rounding_ them, so:
* Line Item 1: `1.193791 * 25 = 29.844775`
* Line Item 2: `1.193791 * 40 = 47.75164`
* Total of summed Line Items: `77.596415`
`CRM_Utils_Money::equals()` then rounds _up_ not _down_ because of the extra precision and :boom: the above params cannot successfully create an Order.
My suggestion is that CiviCRM should:
* Verify the amount in each Line Item to the number of decimal places for the given currency.
* Keep a running total of those amounts.
* Compare the final running total with the `total_amount`.https://lab.civicrm.org/dev/core/-/issues/2915Proposal: Remove the Group Visibility terminology "Expose Publicly" as it has...2024-02-08T23:31:11Zjustinfreeman (Agileware)Proposal: Remove the Group Visibility terminology "Expose Publicly" as it has unintended connotations and would be good to standardise the terminology to describe "Group Visibility" and its optionsThis is a proposal to:
1. Remove the **Group Visibility** terminology _Expose Publicly_ as it has unintended connotations (as pointed out to me today by a Lawyer) and
2. Would be good to standardise the terminology to describe **Group Vi...This is a proposal to:
1. Remove the **Group Visibility** terminology _Expose Publicly_ as it has unintended connotations (as pointed out to me today by a Lawyer) and
2. Would be good to standardise the terminology to describe **Group Visibility** and its options. Ideally making it more obvious as to the purpose of each option.
As described on https://docs.civicrm.org/user/en/latest/organising-your-data/groups-and-tags/#visibility
* **Group Visibility**: **Public pages / Expose Publicly** - if you want to allow contacts to join and remove themselves from this group via Registration and Account Profile forms
* **Group Visibility**: **User and User Admin Only** - if membership in this group is controlled only by authorised CiviCRM users
As I understand it, this option _only_ applies to **Mailing List, Group Types**.
# Expose Publicly
![expose-publicly](/uploads/8d44c054f82a86d8fe92f8f7cd535663/expose-publicly.png)
# Group Visibility and options
![visbility](/uploads/7c4069b330d413230996fab273d242f4/visbility.png)https://lab.civicrm.org/dev/financial/-/issues/188Financial Items incorrectly recorded when using Payment API2021-10-27T16:53:49ZhaystackFinancial Items incorrectly recorded when using Payment APIFinally found the time to return to the Order & Payment API and test the changes that @eileen and @KarinG have worked on. So much better now that I can update a contribution without the tax amounts being recalculated!
Warning, this is g...Finally found the time to return to the Order & Payment API and test the changes that @eileen and @KarinG have worked on. So much better now that I can update a contribution without the tax amounts being recalculated!
Warning, this is going to be a long issue because it seems to involve two (maybe overlapping) issues which I haven't been able to separate. So... the scenario that I'm testing is where:
* I create multiple Line Items in an Order that are a mix of taxable and non-taxable.
* They are also a mix of Participant and Membership Line Items.
* The Line Items can be in any sequence when the Order is created.
To tease out what's going on, I tested the following:
1. Taxable Participant ($25) followed by Non-taxable Membership ($50)
2. Non-taxable Membership ($50) followed by Taxable Participant ($25)
3. Taxable Participant ($50) followed by Non-taxable Membership ($50)
4. Non-taxable Membership ($50) followed by Taxable Participant ($50)
These happen to be for Events ("Summer Solstice Festival Day Concert" and "Fall Fundraiser Dinner") and Membership Types (Student) in the CiviCRM Sample Data - though if you want to use those to replicate then it should be noted that "Summer Solstice Festival Day Concert" is incorrectly set to the "Member Dues" Financial Type. @kcristiano was invaluable in setting up CiviCRM's Sample Data with correct Financial Types and I assume there's nothing amiss in that regard.
## Taxable Participant ($25) followed by Non-taxable Membership ($50)
The procedure is commented upon in full for this first test. I'll just post the annotated logs for the subsequent tests since the procedure is identical.
![civicrm-E25-M50-order](/uploads/74c5bdb91fb8ee46d73a0f525a7552ad/civicrm-E25-M50-order.png)
The params for Order.create are as follows:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 11:28:21
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 79.84 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Solstice Ticket x 1, Student Membership x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[source] => Shop: Solstice Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[label] => Solstice Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
[18] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
)
)
```
The CiviCRM API (yeah, I know I should convert to API4 but that's another story) returns:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011112821
[non_deductible_amount] =>
[total_amount] => 79.844775 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 79.844775 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 25.00
[label] => Solstice Ticket
[line_total] => 25.00
[tax_amount] => 4.844775 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
)
)
)
)
```
So far so good - though it seems odd that the Total Amount, Net Amount and Tax Amount are reported as having 6 decimal places given that a pre-calculated Tax Amount to 2 decimal places was passed in.
Let's have a look at the Bookkeeping Report at this stage:
![civicrm-E25-M50-pre](/uploads/0fa2ceec4ca463ce50a38bb0ff8b96ae/civicrm-E25-M50-pre.png)
Looks good. The database looks good too:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:28:21 | 79.84 | 79.84 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 25.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 4.84 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 50.00 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Submit the `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 79.84
[trxn_date] => 2021-10-11 11:30:58
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
And the logs show:
```
PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
The API result is:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 11:30:58
[total_amount] => 79.84
[fee_amount] => 0.00
[net_amount] => 79.84
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Nice. So what's in the database?
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Solstice Ticket | 25.00 | USD | 15 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:28:21 | 79.84 | 79.84 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 11:30:58 | 79.84 | 79.84 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 |
| 204 | civicrm_financial_item | 104 | 101 | 25.00 |
| 205 | civicrm_financial_item | 105 | 101 | 4.84 |
| 206 | civicrm_financial_item | 106 | 101 | 50.00 |
| 207 | civicrm_contribution | 103 | 102 | 79.84 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 25.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 105 | 102 | 4.84 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 105 | 102 | 4.84 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
The Bookkeeping Report seems to reflect the odd assignment of Financial Items:
![civicrm-E25-M50-post](/uploads/2e550b6a4893a9d8960ec4b0a2bcff14/civicrm-E25-M50-post.png)
As you can see, it lists all of the `1100` entries as "Event Fee Taxable". The "Member Dues" item seems to have been switched to "Event Fee Taxable". There also seems to be a duplicate Tax Amount entry. This may be important below when the Financial Item amounts are identical.
## Non-taxable Membership ($50) followed by Taxable Participant ($25)
Sequence of Line Items is reversed:
![civicrm-M50-E25-order](/uploads/3fb49ce43318c51ba2da2f6cd8be3470/civicrm-M50-E25-order.png)
Order API params:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 11:36:49
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 79.84 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Student Membership x 1, Solstice Ticket x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
[18] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[source] => Shop: Solstice Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[label] => Solstice Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
)
)
```
API returns:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011113649
[non_deductible_amount] =>
[total_amount] => 79.844775 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 79.844775 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 25.00
[label] => Solstice Ticket
[line_total] => 25.00
[tax_amount] => 4.844775 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
)
)
)
)
```
Database looks good:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:36:49 | 79.84 | 79.84 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 50.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 25.00 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 4.84 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Bookkeeping confirms this:
![civicrm-M50-E25-pre](/uploads/87857b96e6fb0b8b2379c29e832da746/civicrm-M50-E25-pre.png)
Submit the `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 79.84
[trxn_date] => 2021-10-11 11:38:59
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
Logs again show:
```
[11-Oct-2021 10:39:00 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
[11-Oct-2021 10:39:01 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
API result:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 11:38:59
[total_amount] => 79.84
[fee_amount] => 0.00
[net_amount] => 79.84
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Database again shows the wrongly assigned Financial Items:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Student Membership | 50.00 | USD | 2 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:36:49 | 79.84 | 79.84 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 11:38:59 | 79.84 | 79.84 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 25.00 |
| 206 | civicrm_financial_item | 106 | 101 | 4.84 |
| 207 | civicrm_contribution | 103 | 102 | 79.84 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 106 | 102 | 4.84 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 25.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 106 | 102 | 4.84 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
Again, there also seems to be a duplicate Tax Amount entry.
Bookkeeping Report confirmation of the odd assignment:
![civicrm-M50-E25-post](/uploads/7124d730e050f3f288de87ed29bfeb1b/civicrm-M50-E25-post.png)
The "Event Fee Taxable" item seems to have been switched to "Member Dues".
## Taxable Participant ($50) followed by Non-taxable Membership ($50)
Sequence as per first test:
![civicrm-E50-M50-order](/uploads/3cadc61cf6ecceeb570c7298ae0aafb5/civicrm-E50-M50-order.png)
Order API params:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 12:01:02
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 109.69 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Fundraiser Dinner Ticket x 1, Student Membership x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 1
[contact_id] => 210
[role_id] => 1
[source] => Shop: Fundraiser Dinner Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 9.69 <-- Note: Tax Amount has 2 decimal places
[label] => Fundraiser Dinner Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
[18] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
)
)
```
API return:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011120102
[non_deductible_amount] =>
[total_amount] => 109.68955 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 109.68955 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 9.69 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 50.00
[label] => Fundraiser Dinner Ticket
[line_total] => 50.00
[tax_amount] => 9.68955 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
)
)
)
)
```
Again, database looks good:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:01:02 | 109.69 | 109.69 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 50.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 9.69 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 50.00 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Bookkeeping Report confirms:
![civicrm-E50-M50-pre](/uploads/728d5845348f0f8bf289f5a7fbcb84f6/civicrm-E50-M50-pre.png)
This is where things get a bit weird(er).
Call `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 109.69
[trxn_date] => 2021-10-11 12:04:49
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
Logs (as usual) show:
```
[11-Oct-2021 11:04:50 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
[11-Oct-2021 11:04:51 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
API result is:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 12:04:49
[total_amount] => 109.69
[fee_amount] => 0.00
[net_amount] => 109.69
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Database has the same pattern of mis-assignment:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:01:02 | 109.69 | 109.69 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 12:04:49 | 109.69 | 109.69 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 9.69 |
| 206 | civicrm_financial_item | 106 | 101 | 50.00 |
| 207 | civicrm_contribution | 103 | 102 | 109.69 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 105 | 102 | 9.69 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 105 | 102 | 9.69 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
However, this time the consequences of this combination of mismatch and duplicate entry gives a different result in the Bookkeeping Report:
![civicrm-E50-M50-post](/uploads/afb612896bb51e3db2afd7b67b176c9d/civicrm-E50-M50-post.png)
This time, it seems, there are only 5 Financial Items visible.
## Non-taxable Membership ($50) followed by Taxable Participant ($50)
![civicrm-M50-E50-order](/uploads/0f34a882ab9bfb12322254183951b89a/civicrm-M50-E50-order.png)
I'll skip the API calls for this one - if you want them, I'm happy to provide.
The `Order.create` process goes fine, but we're left with the following in the database:
```
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Student Membership | 50.00 | USD | 2 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:21:31 | 109.69 | 109.69 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 12:25:19 | 109.69 | 109.69 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 50.00 |
| 206 | civicrm_financial_item | 106 | 101 | 9.69 |
| 207 | civicrm_contribution | 103 | 102 | 109.69 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 106 | 102 | 9.69 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 106 | 102 | 9.69 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
The Bookkeeping Report shows the end result:
![civicrm-M50-E50-post](/uploads/4cefa4c34c1b02956fdc9ac3e62c8d4c/civicrm-M50-E50-post.png)
## Final thought
My best guess at the moment is that the combination of the mis-assigned `entity_id`s for the Financial Items _plus_ the effect of the extra entry in the `civicrm_entity_financial_trxn` table leads to the mix of 5 or 6 Items visible in the Bookkeeping Report.
If you've got to here, thanks for reading! Hope this helps fix what seemed to be mind-bending symptoms.https://lab.civicrm.org/dev/core/-/issues/2899DB error on updating multiple cases from the search form2022-09-20T10:50:03ZjitendraDB error on updating multiple cases from the search formTo replicate -
1. Navigate to https://dmaster.demo.civicrm.org/civicrm/case/search?reset=1
2. Search for cases. Select all results (ts_all radio button) => Update multiple cases action
![image](/uploads/fcbb716228f064c5cde2f8231a534e15...To replicate -
1. Navigate to https://dmaster.demo.civicrm.org/civicrm/case/search?reset=1
2. Search for cases. Select all results (ts_all radio button) => Update multiple cases action
![image](/uploads/fcbb716228f064c5cde2f8231a534e15/image.png)
3. Select Profile on the next step and hit Continue;
![image](/uploads/cb5fb171efd98f5014e5c822f367f717/image.png)https://lab.civicrm.org/dev/core/-/issues/2869Api4 explorer - Clicking the Debug checkbox on or off doesn't change the samp...2023-09-30T05:15:09ZDaveDApi4 explorer - Clicking the Debug checkbox on or off doesn't change the sample codeI think it should add setDebug()?
Issue needs debugging. I tried clicking the debug checkbox to ... oh, right.
FYI @colemanwI think it should add setDebug()?
Issue needs debugging. I tried clicking the debug checkbox to ... oh, right.
FYI @colemanwhttps://lab.civicrm.org/dev/core/-/issues/2864Meta - token usage 5.43 standardisation effort2022-09-02T01:38:26ZeileenMeta - token usage 5.43 standardisation effortWe are doing a big token standardisation in 5.43 to solve multiple issues and blockers. By getting all these changes into 1 release we can also encourage people to specifically test that rc and any changes that they may need to make can ...We are doing a big token standardisation in 5.43 to solve multiple issues and blockers. By getting all these changes into 1 release we can also encourage people to specifically test that rc and any changes that they may need to make can be around the one release - documentation is [here](https://lab.civicrm.org/-/ide/project/documentation/docs/sysadmin/tree/case/-/docs/upgrade/version-specific.md/)
The goal is to consolidate such that
1. All token rendering is done through the token processor, token hooks are redundant and deprecated
2. All functions on CRM_Utils_Token are unused in core and deprecated except for those in the legacy BAO mailing classes.
3. We freeze, deprecate and phase out in legacy BAO classes in favour of flexmailer
4. All core classes that render tokens do so using consistent token naming & output conventions (rather than dates being formatted in various ways and fields sometimes outputting labels and other times values or name fields https://lab.civicrm.org/dev/core/-/issues/2650)
5. As much as possible we render via tokens rather than smarty in workflow message templates (that couples them with the entity rather than any form flow)
6. pdf & email & scheduled reminder tasks consistently offer the same entity-specific, contact & domain tokens
7. We can start to define obvious missing tokens such as `{domain.now}` (done), `{contribution.balance}` and formatted addresses
Task list / streams - this gives some idea of where things are progressing or blocked
- [x] standardise contact tokens to support/ advertise label style (e.g `{contact.prefix_id:label}`
- [x] make Contact tokens work with partial contact inputs - currently they only work with no contact array passed in, or an array with all possible values - currently blocked on a caching bug https://github.com/civicrm/civicrm-core/pull/21790
- [x] split contact tokens into own class & extend entity tokens like the other classes(blocked on stdise contact tokens)
- [x] switch contact tokens to use apiv4
- [x] work through standardising location token loading (this could go further - ie I had thought about adding loading for billing
- [x] date formatting standardisation [documentation](https://docs.civicrm.org/user/en/latest/common-workflows/tokens-and-mail-merge/#date) https://lab.civicrm.org/dev/core/-/issues/2746
- [x] locale sensitive standardised money formatting https://lab.civicrm.org/dev/core/-/issues/2638 & [pr](https://github.com/civicrm/civicrm-core/pull/21783)
- [x] switch badge tokens to use token processor
- [x] event tokens cleanup & loading fixes + remove unindexed query, move the `fee_amount` and `balance` tokens to the participant tokens class, fix event class to listen to either 'participantId' or 'eventId'. Participant class should run first & load eventId if required. Events should be cached in a static cache as it would be very rare to message thousands of events in one run.
- [x] Recurring contribution tokens work (currently availble in only the recurring_edit template with a pr to extend to cancelled recurring https://test.civicrm.org/job/CiviCRM-Core-PR/44632/
- [x] pdf letter is sane (https://lab.civicrm.org/dev/core/-/issues/2790) enough to extend to deliver tokens for other entities (participant https://lab.civicrm.org/dev/core/-/issues/780)
- [x] email letter extend to deliver tokens for other entities (membership & participant) (https://lab.civicrm.org/dev/core/-/issues/2862)
- [x] scheduled reminders supports participant tokens https://lab.civicrm.org/dev/core/-/issues/779
- [x] domain tokens are consistently available & render https://lab.civicrm.org/dev/core/-/issues/2838 - done but a gap on case tasks as they prioritise being able to filter tokens by case type
- [x] install flexmailer on new installs https://lab.civicrm.org/dev/core/-/issues/2836
- [ ] fix remaining places that process tokens other than via the payment processor
- [x] eliminate replaceGreeting - currently [blocked on apiv4 caching bug](https://github.com/civicrm/civicrm-core/pull/21790)
- [ ] fix CRM_Contact_BAO_Contact_Utils::updateGreeting to not call `getTokenDetails`
- [ ] confirm civicrm_api3_mailing_preview is replaced by flexmailer
- [x] remove getTokenDetails call from CRM_Contribute_Form_Task_PDFLetter - note this appears to be an expensive way to call the contact api & tokens are unused https://github.com/civicrm/civicrm-core/pull/21805
- [x] remove extra rendering in BAO_Pledge https://github.com/civicrm/civicrm-core/pull/21789
- [x] removed getTokenDetails from updatePledgeStatus https://github.com/civicrm/civicrm-core/pull/21847
- [x] remove getTokenDetails from transitionParticipants
- [ ] remove getTokenDetails from sendCancellation & cancelParticipant in selfsvc flow
- [ ] deliverGroup - replaced by flexmailer?
- [ ] MailingPage::preview - replaced by flexmailer?
- [ ] remove hook:;tokenValues from label task
- [ ] getAnonymousTokenDetails ???
- [ ] confirm BAO_Mailing::compose is only via flexmailer
- [x] communicate rc status via dev list
- [x] communicate changes via blog
- [ ] upgrade message https://lab.civicrm.org/dev/core/-/issues/2871
- [ ] docs update to these instructions / check it can still all be done https://docs.civicrm.org/dev/en/latest/step-by-step/create-custom-case-token/
**Currenly out of scope**
- [ ] Token.getlist api - until we have this we have an issue with audience-filtering and with switching tokens depending on the schema. Also case tasks still have to render tokens weirdly without this because they currently filter by case type with partial success https://lab.civicrm.org/dev/core/-/issues/2788
- [ ] Clarify how extensions should render tokens https://lab.civicrm.org/dev/core/-/issues/2863
- [ ] encourage flexmailer with status checks on existing installs https://lab.civicrm.org/dev/core/-/issues/2836
Analysis of current token rendering it core:
Method | Used in | Status
-- | -- | --
Use the token processor | scheduled reminders, civimail with flexmailer in use, activity pdf task | This is the method we are consolidating all paths to use.
Use the functions in CRM_Utils_Token | PDF tasks other than activity | Migrated (PRs on [membership](https://github.com/civicrm/civicrm-core/pull/21521) and [contribution](https://github.com/civicrm/civicrm-core/pull/21524). Also work on fixing up the [pdf task class](https://lab.civicrm.org/dev/core/-/issues/2790) supporting[add participant tokens](https://lab.civicrm.org/dev/core/-/issues/780) (once we’ve finished [the standardisation of those](https://github.com/civicrm/civicrm-core/pull/21587))
| Email tasks | Migrated to token processor. Added tokens for other entities (participant, membership https://lab.civicrm.org/dev/core/-/issues/1396) [also see](https://lab.civicrm.org/dev/core/-/issues/2862)
| Export tasks (when merging contacts in export) | Migrated to token processor
| Profile link rendering | Migrated to token processor
| Storing contact greetings (eg. email_greeting_display) | In progress. Currently resolving inconsistent token syntax and load issues as this needs to be able to use partial pre-loads
Use random ad hoc code | Event badges | [migrated](https://github.com/civicrm/civicrm-core/pull/21587) - this is the only place outside of scheduled reminders where event or participant tokens were used in core
| Send SMS task (BAO_Activity::sendSMS) | migrated
| Labels task CRM_Contact_Form_Task_Label | argh
| CRM_Event_Form_SelfSvcTransfer | Still pretty argh - but this is a workflow message template so the fix is a bit different
| Transition participants | Also a workflow - possibly redundant as seems to be mostly about contact tokens which are resolved in the send function anyway
| Pledge acknowledgements | Also appear to be redundant
| CiviMail with flexmailer not used (many functions & classes) | Push the switch to flexmailer.
| Various unsubscribe / resubscribe type actions | Probably out of scope for this roundhttps://lab.civicrm.org/dev/core/-/issues/2863Agree / document how tokens should be rendered from outside core2023-09-24T22:52:19ZeileenAgree / document how tokens should be rendered from outside coreIdeally we would have one recommended api way to do this. This would be a way that is usable via api explorer and all the ways we use the api and core code would be converted to do things using that same api method
There is [a documenta...Ideally we would have one recommended api way to do this. This would be a way that is usable via api explorer and all the ways we use the api and core code would be converted to do things using that same api method
There is [a documentation pr here](https://lab.civicrm.org/documentation/docs/dev/-/merge_requests/962) - at the moment I don't think it meets this need.
**Permissions & non-workflow rendering**
There is a question as to whether the api would support non-workflow messaging (in which case the api would likely be more like `Message::render()`. The argument against is that the permissions are more complex. However some notes on that
1) the most common use case is probably when checkPermissions = FALSE in php code.
2) there is an appropriate permission already in core
```
render templates' => [
$prefix . ts('render templates'),
ts('Render open-ended template content. (Additional constraints may apply to autoloaded records and specific notations.)'),
],
```
3) currently the v3 MessageTemplate.send api allows access to the core function - it required a messageTemplateID to be set - but this can probably be faked to allow 'any text'https://lab.civicrm.org/dev/core/-/issues/2848Activity tokens - case id - explain?2023-09-24T22:52:50ZeileenActivity tokens - case id - explain?@ayduns I recently fixed up some activity tokens that weren't working - but in the process @DaveD queried whether 'case_id' makes sense as a token on activity tokens since there could be more than one - just opening this put that to you....@ayduns I recently fixed up some activity tokens that weren't working - but in the process @DaveD queried whether 'case_id' makes sense as a token on activity tokens since there could be more than one - just opening this put that to you....
https://github.com/civicrm/civicrm-core/pull/21489#pullrequestreview-757039309https://lab.civicrm.org/dev/core/-/issues/2846Improve validation of start and end dates for events, contribution pages and ...2023-11-23T07:09:32ZJKingsnorthImprove validation of start and end dates for events, contribution pages and more**Motivation**
There are holes in the validation of start and end dates on contribution pages.
eg:
- Create a new contribution page
- Enter a start date
- Enter an end 'time' but leave the date part empty:
![image](/uploads/e75411e618...**Motivation**
There are holes in the validation of start and end dates on contribution pages.
eg:
- Create a new contribution page
- Enter a start date
- Enter an end 'time' but leave the date part empty:
![image](/uploads/e75411e61813ee5ae55b808f49fa3637/image.png)
- Result is a fatal DB error.
This may seem like an edge-case, but it can also happen when you are 'scrolling' down the page, and accidentally 'scroll' on the time field. This sets a time.
The same is true for event pages event 'start' and 'end' dates.
The same is true for event registration open and close dates:
![image](/uploads/270044386b6b0895da079268d9147e8c/image.png)
The same is true for price fields, with the 'active on' and 'expire on', where setting a time without a date results in a fatal error and the endlessly spinning civi icon of doom.
And for price fields, there is currently no validation to ensure the 'expire on' is before the 'active on'.
Creating campaigns also:
![image](/uploads/c51922aa11942eafa7e58e6d153a5fae/image.png)
**Proposed solution**
I think the validation should really be in the date picker element somewhere, to ensure a date and time are both set - where we expect it.
But I also thought we could take this opportunity to standardise the validation of start and end dates in:
- Event pages (event start and end)
- Event pages (registration open/close)
- Contribution pages (start and end)
- Price fields (active on, expire on)
- Campaign (start, end dates)
I am working on this now.
**After**
![image](/uploads/968b1bafab430a8c7cbc2e5a684e1b01/image.png)
**Follow-up / part 2**
Validation at the API/BAO level?https://lab.civicrm.org/dev/financial/-/issues/186Accounting entries incorrect in a number of cases... especially with pending ...2023-05-01T17:18:33ZJamie Novick - CompucoAccounting entries incorrect in a number of cases... especially with pending refunds and overpaymentsFirstly - sorry for the wall of text. This has ended up being a much longer piece of analysis than expected but I thought I should share it so that we can agree a way ahead.
This all started as we were looking at the best way to impleme...Firstly - sorry for the wall of text. This has ended up being a much longer piece of analysis than expected but I thought I should share it so that we can agree a way ahead.
This all started as we were looking at the best way to implement refunds for a payment processor. Some work was done here to discuss this: https://lab.civicrm.org/dev/financial/-/issues/87 but I can see this stalled somewhat - probably due to some of the issues I'm going to discuss here.
# Background: The problems with the contribution status field
Just for complete disclosure I hate the CiviCRM contribution status field. Hopefully everyone already knows that. I've probably started discussions on getting rid of it at every CiviCON and documented this on every financial gitlab ticket. It is the biggest blocker we have to reaching an accounting implementation that aligns to standard concepts of accounting.
Why is that?
We appear to have 2 use cases for it, both of which are undocumented.
1. For users to quickly change whether the amounts owed are
a) expected to be paid or
b) are paid.
2. To reflect the payment status of the contribution
i.e. whether the total sum of the line items is >, < or = to the net amount paid in or refunded on that contribution (this is important and I’ll come back to it)
This is a bit of a problem as it’s all a bit chicken and egg! If the user changes the status we need to understand whether we should be creating payments or not (or creating refund payments or not), and if the user records payments or refund payments we need to update the status to something meaningful and consistent.
The following is an analysis of the allowed contribution status changes from and to that a user can perform:
**Table 1:**
- Black = Option hidden
- X - Val = Option shown but user cannot change to it as we have a validation message
- Y (black) = Option shown, user can select this and I don’t see issues with accounting that occurs
- N/A = Couldn’t test
- Y (red) = Option shown, user can select this and there are issues with accounting that occurs
![Screenshot_2021-09-15_at_15.33.12](/uploads/8db825597f091f0bf34d590b38b2ed2f/Screenshot_2021-09-15_at_15.33.12.png)
I’d note that it’s a bit confusing to users to see a value in the dropdown they can’t use. I think we should just hide values they cannot select as this would be much clearer for them.
We end up with some very strange and in some cases incorrect behaviour when allowing users to change the status in a "partially paid" and "pending refund" scenarios:
**Table 2:**
![Screenshot_2021-09-15_at_15.47.11](/uploads/827aafd531d7dbe54866b8237758ce42/Screenshot_2021-09-15_at_15.47.11.png)
# Ground Rules
As such I would like to suggest some ground rules to work towards:
1. The only time a user can change the status of a contribution should be when it has no payments or refunds attached to it.
2. Once a payment has been received we manage everything through either the line item editor or the “change selections” on the event form or "record payment” or “record refund”. If any changes to each are made to any of these we recalculate the contribution status based on one set of rules:
The rules to calculate the contribution status should be:
**Table 3:**
![Screenshot_2021-09-15_at_15.38.43](/uploads/b77e518446862474f4e6823e045e11a3/Screenshot_2021-09-15_at_15.38.43.png)
Delving deeper, there are a number of situations within CiviCRM that do not confirm to this currently and instead we end up with a status this is confusing for the user:
(Note this is not be a comprehensive list - just to give some examples).
**Table 4:**
![Screenshot_2021-09-15_at_15.39.45](/uploads/3a9e238f9ec1071de6b056c543c68522/Screenshot_2021-09-15_at_15.39.45.png)
# Specific Scenarios
This all said there are some specific use cases that we need to consider and have appropriate workflows for:
Table 5:![Screenshot_2021-09-15_at_15.40.49](/uploads/a922e2c0f5a52b92adba895c5fd89654/Screenshot_2021-09-15_at_15.40.49.png)
# Actions:
As such I think the actions could be:
1. Agree to the "ground rules" for the contribution status field above so we are all working towards a common goal.
1. I think we should hide values they cannot select as this would be much clearer for them. i.e. all items that are marked as “X-val” in the table above. Perhaps we can refactor the code that show/hides the options to a central place.
1. I think we should change the way the status field is calculated to use the business logic suggested in Table 3 above in all cases. This shouldn’t be something that extensions should take care of but should be calculated any time a contribution haas it’s line items / financial transactions changed. Obviously I don’t know what this means from a code perspective and I assume this would be a significant refactor but I think we could clear up a lot of business logic by doing so.
1. Discuss and agree which of the options from table 5.1 are suitable and then agree the UX for this.
1. So that we can then remove the option to make the status changes as per table 1: 3f, 4e, 4f without degrading users ability to do the things they need.
I actually think if we do these things we can consider CiviCRM’s accounting to be “correct” if not completely intuitive (at least for now).
@mattwire @eileen @KarinG @JoeMurrayhttps://lab.civicrm.org/dev/financial/-/issues/185Contributions created as pending can be later completed in test mode, behavin...2021-09-15T13:25:34ZFrancis (Agileware)Contributions created as pending can be later completed in test mode, behaving like live contributionsContributions created as pending can be later completed in test mode using the `civicrm/contribute/transact` path, behaving like live contributions.
e.g. linked Memberships are moved from Pending to New
1. As an admin, create a Contrib...Contributions created as pending can be later completed in test mode using the `civicrm/contribute/transact` path, behaving like live contributions.
e.g. linked Memberships are moved from Pending to New
1. As an admin, create a Contribution Page that performs "real-time transactions" allows the end user to leave a Contribution in Pending state (either pay later or with a payment processor that defers payment with "incomplete transaction"). For complete coverage, include membership sign up configuration.
2. As a normal user, visit the Contribution Page in live mode and submit details, but do not complete the transaction
3. As the same user, reload the Contribution Page in test mode with the relevant contribution id as URL parameter
4. Complete the transaction using test credit card details
5. As an admin, observe that the contribution has been updated as though it were completed in live mode including relevant changes to any linked entities.
Suggest that this page should probably refuse access in test mode if it attempts to load an existing contribution that is not marked `is_test`https://lab.civicrm.org/dev/core/-/issues/2811Menu Angular error after upgrade2023-01-08T01:26:08ZStoobMenu Angular error after upgradeI do a fair amount of upgrades, and anecdotally after about a _half_ of recent upgrades, the CiviCRM menu fails to load. For half of those, a simple reload/refresh of the browser resolves the issue. For the other half, the civicrm/menu...I do a fair amount of upgrades, and anecdotally after about a _half_ of recent upgrades, the CiviCRM menu fails to load. For half of those, a simple reload/refresh of the browser resolves the issue. For the other half, the civicrm/menu/rebuild cache (or drush cc) must be cleared in order for the menu to appear. It is usually an Angular issue, seen attached.
Two questions, if these steps are required with such frequency can the upgrade process be improved by either:
1. clearing the cache automatically when the upgrade is finished
2. providing instructions and/or link on the upgrade screen to clear cache
![menu-load](/uploads/70b9bc8bcfbae067b5169d330bd16545/menu-load.png)
![upg](/uploads/19aa40afb6bda323d97cafe3c3cec5b3/upg.png)https://lab.civicrm.org/dev/core/-/issues/2796Regression? No way to change the tax rate and tax amount using buildamount hook2023-11-24T22:45:42ZPradeep Nayakpradpnayak@gmail.comRegression? No way to change the tax rate and tax amount using buildamount hookIn 5.35.x LineItem create api allowed one to set a different tax_amount in api params for a financial type that had tax enabled. The tax amount could then be calculated programatically based on region, location etc instead of using the c...In 5.35.x LineItem create api allowed one to set a different tax_amount in api params for a financial type that had tax enabled. The tax amount could then be calculated programatically based on region, location etc instead of using the config specified in the UI.
```
civicrm_api3('LineItem', 'create', [
'entity_id' => 284,
'qty' => 1,
'unit_price' => 100,
'line_total' => 100,
'entity_table' => "civicrm_contribution",
'tax_amount' => 20,
'financial_type_id' => "Donation",
'price_field_id' => "contribution_amount",
'price_field_value_id' => "single",
]);
```
Another way of calculating tax on online forms was using the [buildamount hook](https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_buildAmount/)
```
function advanceinvoicing_civicrm_buildAmount($pageType, &$form, &$amount) {
$amount[$priceFieldId]['options'][$priceFieldValueId]['tax_rate'] = 10;
$amount[$priceFieldId]['options'][$priceFieldValueId]['tax_amount'] = 0.1;
}
```
A recent [change](https://github.com/civicrm/civicrm-core/commit/e967ce8fe2b#diff-a16d4d7449cf5f3a0616d1d282a32f27ab6d3f7d2726d076c02ad1d4d655af41R56) to the line item create function has stopped allowing tax amount to be set programmatically.
Is this intentional? Is there any way to support different tax rate for single financial type?https://lab.civicrm.org/dev/core/-/issues/2795PHP8: Use of error-suppression operator (@) behaves differently in error_hand...2023-12-12T19:51:20ZDaveDPHP8: Use of error-suppression operator (@) behaves differently in error_handler functions in php8Parking what's left in https://github.com/civicrm/civicrm-core/pull/21064 here for now since I've cleared out a bunch and am taking a break from it. The gist is that in php8 the error suppression operator (e.g. `@fopen('/nonexistent/file...Parking what's left in https://github.com/civicrm/civicrm-core/pull/21064 here for now since I've cleared out a bunch and am taking a break from it. The gist is that in php8 the error suppression operator (e.g. `@fopen('/nonexistent/file.txt', 'r')` works differently with respect to error_handler functions. In a stock install as of right now you won't notice any difference, just there will be more errors depending on what 3rd party/custom error_handlers you're using and may behave differently on different CMSs in future depending on how their error-handling may change.
### smarty default modifier
The smarty modifier `|default` is a tricky one because its whole purpose is to deal with the situation where the var might be missing, but the way smarty compiles it is something like `<?php $this->assign('to', ((is_array($_tmp=@$this->_tpl_vars['to'])) ? $this->_run_mod_handler('default', true, $_tmp, '_high') : smarty_modifier_default($_tmp, '_high'))); ?>` which uses the @ to try to silence the error. In addition to [templates/CRM/Core/DatePickerRange.tpl and templates/CRM/Core/DatePickerRangeWrapper.tpl](https://github.com/civicrm/civicrm-core/commit/f671f500b658aa772fedc682dbb320cf456bc904), the below are all instances of this. They could all be rewritten to not use the modifier, but the modifier itself is part of smarty so can't easily be prevented from being used in future.
* CRM_Core_InvokeTest::testInvokeDashboardForNonAdmin
* CRM/Contact/Page/DashboardDashlet.tpl - $communityMessages
* CRM_Core_SessionTest::testSetStatusWithTextOnly
* CRM/common/info.tpl
* CRM_Core_SessionTest::testSetStatusWithTitleOnly
* CRM/common/info.tpl
* CRM_Core_SessionTest::testSetStatusWithBoth
* CRM/common/info.tpl
### opendir
```
CRM_Utils_File::cleanDir
if ($dh = @opendir($target)) {
```
### unlink and mkdir
Various calls to `@unlink` and `@mkdir`
### httpclient
```
CRM_Utils_HttpClientTest::testFetchHttp_badOutFile
Exception: fopen(/ba/d/path/too/utput): Failed to open stream: No such file or directory
...\tests\phpunit\CiviTest\CiviUnitTestCase.php:253
...\CRM\Utils\HttpClient.php:79 <---- see here
...\tests\phpunit\CRM\Utils\HttpClientTest.php:77
...\tests\phpunit\CiviTest\CiviUnitTestCase.php:267
...\phpunit8:671
```
### angular changeset
Aside: The code in question here suggests phpQuery has its own debug setting which maybe could be turned on automatically in future if you have civi debugging enabled:
`$return = phpQuery::$debug === 2 ? this->document->loadHTML($markup) : @$this->document->loadHTML($markup);`
```
Civi\Angular\ChangeSetTest::testInsertAfter
Exception: DOMDocument::loadHTML(): htmlParseEntityRef: no name in Entity, line: 2
...\tests\phpunit\CiviTest\CiviUnitTestCase.php:253
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\DOMDocumentWrapper.php:200
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\DOMDocumentWrapper.php:542
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\DOMDocumentWrapper.php:505
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\DOMDocumentWrapper.php:469
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\phpQueryObject.php:2066
...\vendor\electrolinux\phpquery\phpQuery\phpQuery\phpQueryObject.php:2010
...\tests\phpunit\Civi\Angular\ChangeSetTest.php:28
...\Civi\Angular\ChangeSet.php:59
...\Civi\Angular\ChangeSet.php:19
...\tests\phpunit\Civi\Angular\ChangeSetTest.php:39
...\tests\phpunit\CiviTest\CiviUnitTestCase.php:267
...\phpunit8:671
```
* not ok 103 - Error: Civi\Angular\PartialSyntaxTest::testConsistencyExamples with data set 6 `('<div ng-if="a && b"></div>', '<div ng-if="a && b"></div>')`
* not ok 104 - Error: Civi\Angular\PartialSyntaxTest::testAllPartials
* These are the same issue as the changeset tests.
### testing of deprecations
These deliberately test deprecations
* not ok 223 - Error: Civi\Payment\PropertyBagTest::testSetContactIDLegacyWay
* not ok 227 - Error: Civi\Payment\PropertyBagTest::testSetCustomProp
### testGetfieldsHasTitle
```
api_v3_SyntaxConformanceTest::testGetfieldsHasTitle with data set 32 ('Entity')
Failure in api call for Entity getactions: include_once(api/v3/batch_create.php): failed to open stream: No such file or directory
0 ...\api\v3\Entity.php(18): CiviUnitTestCase->letssee(2, 'include_once(ap...', '\path\to\...', 18, Array)
1 ...\api\v3\Entity.php(18): include_once()
2 ...\api\v3\utils.php(2461): _civicrm_api3_entity_deprecation(Array)
3 ...\api\v3\utils.php(251): _civicrm_api3_deprecation_check('Entity', Array)
4 ...\Civi\API\Provider\ReflectionProvider.php(108): civicrm_api3_create_success(Array, Array, 'Entity', 'getactions')
5 ...\Civi\API\Kernel.php(149): Civi\API\Provider\ReflectionProvider->invoke(Array)
6 ...\Civi\API\Kernel.php(81): Civi\API\Kernel->runRequest(Array)
7 ...\api\api.php(22): Civi\API\Kernel->runSafe('Entity', 'getactions', Array)
8 ...\Civi\Test\Api3TestTrait.php(292): civicrm_api('Entity', 'getactions', Array)
9 ...\Civi\Test\Api3TestTrait.php(173): CiviUnitTestCase->civicrm_api('Entity', 'getactions', Array)
10 ...\tests\phpunit\api\v3\SyntaxConformanceTest.php(1662): CiviUnitTestCase->callAPISuccess('Entity', 'getactions', Array)
11 phar://.../phpunit8/phpunit/Framework/TestCase.php(1213): api_v3_SyntaxConformanceTest->testGetfieldsHasTitle('Entity')
12 ...\tests\phpunit\CiviTest\CiviUnitTestCase.php(267): PHPUnit\Framework\TestCase->runTest()
13 phar://.../phpunit8/phpunit/Framework/TestCase.php(889): CiviUnitTestCase->runTest()
14 phar://.../phpunit8/phpunit/Framework/TestResult.php(577): PHPUnit\Framework\TestCase->runBare()
15 phar://.../phpunit8/phpunit/Framework/TestCase.php(663): PHPUnit\Framework\TestResult->run(Object(api_v3_SyntaxConformanceTest))
16 phar://.../phpunit8/phpunit/Framework/TestSuite.php(481): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
17 phar://.../phpunit8/phpunit/Framework/TestSuite.php(481): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
18 phar://.../phpunit8/phpunit/TextUI/TestRunner.php(462): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
19 phar://.../phpunit8/phpunit/TextUI/Command.php(129): PHPUnit\TextUI\TestRunner->doRun(Object(PHPUnit\Framework\TestSuite), Array, Array, true)
20 phar://.../phpunit8/phpunit/TextUI/Command.php(101): PHPUnit\TextUI\Command->run(Array, true)
21 ...\phpunit8(671): PHPUnit\TextUI\Command::main()
22 {main}
Failed asserting that a integer is empty.
...\Civi\Test\Api3TestTrait.php:110
...\Civi\Test\Api3TestTrait.php:174
...\tests\phpunit\api\v3\SyntaxConformanceTest.php:1662
...\tests\phpunit\CiviTest\CiviUnitTestCase.php:267
...\phpunit8:671
```
### mailing testConcurrency
This doesn't run properly on windows even without the error_handler, and I don't feel like taking the time right now to deal with that:
not ok 1161 - Error: api_v3_JobProcessMailingTest::testConcurrency with data set #0 (array(20, 3, 10, 4, 1), array(2, 1), 4)