Development issueshttps://lab.civicrm.org/groups/dev/-/issues2020-11-09T13:37:51Zhttps://lab.civicrm.org/dev/financial/-/issues/155Incorrect deductible shown in contribution summary report2020-11-09T13:37:51ZMonish DebIncorrect deductible shown in contribution summary reportSteps to replicate:
1. Ensure that there are multiple contributions with non-zero non-deductible amount.
2. Go to Contribution Summary Report
3. Select Non-Deductible amount column + Grouping on Receive Date - Month
Result: Total non-d...Steps to replicate:
1. Ensure that there are multiple contributions with non-zero non-deductible amount.
2. Go to Contribution Summary Report
3. Select Non-Deductible amount column + Grouping on Receive Date - Month
Result: Total non-deductible amount is incorrect against each month
On closer look, group function (- SUM) is not applied on non-deductible amount, thus the report column show random amount.
Here's a screencast that explains the issue, where I added two contributions this month w/o non-deductible amount respectively, but the Contribution summary report shows incorrect value:
![yhv](/uploads/e851d5c3b56e46c1de049d12d1f977ff/yhv.gif)
ping @JoeMurray @eileen @mattwire @EdselopezMonish DebMonish Debhttps://lab.civicrm.org/dev/financial/-/issues/157Proposal - if payment_processor_type does not define url_site_default then do...2020-12-04T08:47:08ZeileenProposal - if payment_processor_type does not define url_site_default then do not present url_site on Processor formWhen you go to create a new processor there is a field for the site_url (& some other urls) which I generally don't think it's good to have users fill in and would rather suppress. I think it should be suppressed based on payment process...When you go to create a new processor there is a field for the site_url (& some other urls) which I generally don't think it's good to have users fill in and would rather suppress. I think it should be suppressed based on payment processor & while I could make a case for using a supportsUserDefinedUrl method I think we could get away with either
1) don't display the field if payment_processor.site_url_default is empty or
2) don't display the field is payment_processor.site_url_default is 'null' or some other magic value
We aren't currently doing any 'supports' checks on the form & it feels like this is probably something used primarily by older processors so I lean towards 1 being enough
@mattwire @KarinG @adixon @artfulrobot @andrewhunt thoughtshttps://lab.civicrm.org/dev/core/-/issues/2266Wording beside on-behalf-of checkbox isn't required, but has no default eithe...2023-11-23T07:25:09ZDaveDWording beside on-behalf-of checkbox isn't required, but has no default either, so appears as a mysterious checkboxI'm not sure yet if this is recent. I remember there being a default but it's not a strong memory.
1. Create a contribution page.
1. On the title page there's a choice to "Allow individuals to contribute and / or signup for membership o...I'm not sure yet if this is recent. I remember there being a default but it's not a strong memory.
1. Create a contribution page.
1. On the title page there's a choice to "Allow individuals to contribute and / or signup for membership on behalf of an organization". Check it.
1. In the section that appears there's a box for a label that has no default and is not required. Leave it blank.
1. Finish off the rest of the settings - doesn't matter too much what.
1. Visit the contribution page. It looks like this:
![Untitled](/uploads/4776b6636a1ea3fb9106face184bd545/Untitled.png)
It's easily fixable by adding words in the label box, but it just a seems a wrong default and I don't remember it coming up as a problem before.
I notice the label box is a textarea for some reason. Maybe it used to be a normal textbox and so the default now isn't getting set correctly?
Also ignore the decimal places in the dollar amounts in the screenshot - that's a known windows issue with civi.https://lab.civicrm.org/dev/core/-/issues/2414Deductible value treated as non-deductible if a non-deductible selection made...2023-11-23T07:09:32Zfreeform.stephDeductible value treated as non-deductible if a non-deductible selection made in the same contributionOriginally reported here: https://civicrm.stackexchange.com/questions/35284/non-deductible-value-incorrect-if-you-choose-a-nondeductible-membership-type-and
Possibly related to: https://lab.civicrm.org/dev/core/-/issues/1083
If you add...Originally reported here: https://civicrm.stackexchange.com/questions/35284/non-deductible-value-incorrect-if-you-choose-a-nondeductible-membership-type-and
Possibly related to: https://lab.civicrm.org/dev/core/-/issues/1083
If you add a donation field (financial type 'donation' which is deductible) to a membership price set that includes membership types that are nondeductible, and then use that price set for a membership contribution form, the non-deductible value on the contribution is saved as membership fee + donation amount instead of just the membership fee portion of the contribution. This means the non-deductible value can not be trusted.
We use the CDN Tax Receipts extension so to get around the issue we wiped out the nondeductible value from all relevant contributions and let the extension + a custom extension calculate the correct values.
How to replicate:
- add a test organization and test individual
- create a non-deductible financial type and a deductible financial type
- create 2 new membership types for your test organization with a fee and set one to the deductible type, and one to the non-deductible type
- create a price set with 2 fields:
- radio option with the 2 membership types
- radio option with a couple of donation values to choose from, financial type 'donation'
- create a new contribution form, default financial type your custom deductible financial type, and enable your price set on the memberships tab
- now use your new form to create a membership for your test individual and make sure to select the nondeductible membership type & a donation value.
- go to the individual's profile and pull-up the contribution on the contributions tab
To me, it would make sense if the non-deductible value included the donation amount if the contribution status were anything other than 'complete' (pending check for example, because we wouldn't want to issue a tax receipt for monies not receive), but once that status is set to complete then the nondeductible value should respect the financial types of the individual line items.![3X9oo](/uploads/71f07a5b7390c453e28a7ba9f94c54b6/3X9oo.png)seamusleeseamusleehttps://lab.civicrm.org/dev/core/-/issues/3218CSV exports does not respect localization configuration2023-10-19T14:38:19ZmarcusmCSV exports does not respect localization configurationWhen creating CSV exports from searches the e. g. money or date fields are not exported with the configured localization formats.
Configure language: German, decimal separator: ",", thousands sep.: "."
The export looks like this:
```...When creating CSV exports from searches the e. g. money or date fields are not exported with the configured localization formats.
Configure language: German, decimal separator: ",", thousands sep.: "."
The export looks like this:
```
| Nettobetrag |
|-------------|
| 135.00 |
| 20.00 |
| 50.00 |
| 1,130.00 |
should be
| Nettobetrag |
|-------------|
| 135,00 |
| 20,00 |
| 50,00 |
| 1.130,00 |
```
and an example for a date field:
```
| Geburtsdatum |
|--------------|
| 1942-05-02 |
| 1983-06-01 |
should be
| Geburtsdatum |
|--------------|
| 02.05.1942 |
| 01.06.1983 |
```
The export was created with a contribution search.https://lab.civicrm.org/dev/core/-/issues/3204Add recurring contributions to contribution reports2022-09-16T21:11:08ZlarsssandergreenAdd recurring contributions to contribution reportsIt would be very convenient to be able to do filter by recurring contributions for reports, add recurring contributions as a column or even group by.
[Here's a PR that adds this for the Contribution Summary report.](https://github.com/c...It would be very convenient to be able to do filter by recurring contributions for reports, add recurring contributions as a column or even group by.
[Here's a PR that adds this for the Contribution Summary report.](https://github.com/civicrm/civicrm-core/pull/20168) I will also implement in other relevant reports once this has been reviewed.https://lab.civicrm.org/dev/core/-/issues/3314Contribution pages with memberships don't validate correctly when using "quic...2024-02-27T01:10:05ZJonGoldContribution pages with memberships don't validate correctly when using "quick config" price fieldsIn `CRM_Contribute_Form_Contribution_Main::formRule`, there are multiple references to `$self->_useForMember`. `$self->_useForMember` is only `TRUE` if the price set is a "quick config" price set (and is explicitly used as such in the t...In `CRM_Contribute_Form_Contribution_Main::formRule`, there are multiple references to `$self->_useForMember`. `$self->_useForMember` is only `TRUE` if the price set is a "quick config" price set (and is explicitly used as such in the template). However, during `formRule` validation, it's used as a proxy for "this contribution page uses memberships". @mattwire correctly identified the problem on the confirmation class in [PR 15094](https://github.com/civicrm/civicrm-core/pull/15094).
As a result, several membership-specific validations are bypassed.
I attempted to move his methods to the shared `ContributionBase` class and make them protected, but I ran into deeper validation issues. I can't justify further work on this, but wanted to document for future folks who bang their heads against this.https://lab.civicrm.org/dev/financial/-/issues/175Can't remove previously-added currencies2021-07-01T18:10:13Zmegaphonetech-dennisCan't remove previously-added currenciesOverview
----------------------------------------
When I remove previously-added currencies from the list of available currencies under *Language and Currency* and hit the **Save** button, the currencies are still listed among the avail...Overview
----------------------------------------
When I remove previously-added currencies from the list of available currencies under *Language and Currency* and hit the **Save** button, the currencies are still listed among the available currencies when I return to that page.
Reproduction steps
----------------------------------------
1. Go to **Administer** >> **Localization** >> **Languages, Currency, Locations**
1. Next to *Available Currencies* add some currencies. I used CLP and TWD. Click on the **Save** button located at the bottom of the page.
1. Return to **Administer** >> **Localization** >> **Languages, Currency, Locations**
1. Next to *Available Currencies* remove previously-added currencies, and click on the **Save** button located at the bottom of the page.
1. Return to **Administer** >> **Localization** >> **Languages, Currency, Locations** and find those currencies still listed next to *Available Currencies*.
Current behaviour
----------------------------------------
There are no error messages or any indication that something has gone wrong.
Expected behaviour
----------------------------------------
I expected the currencies I removed from this page to no longer be listed as *Available Currencies* when I returned to this page.
Environment information
----------------------------------------
Replicable on the demo server.https://lab.civicrm.org/dev/financial/-/issues/176Billing fields aren't hidden even when they can2024-02-19T16:44:11ZcapoBilling fields aren't hidden even when they canThe **assignAddressField** function, implemented in [CRM/Core/BAO/UFField.php#L785](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/UFField.php#L785), is meant to return a boolean that tells if:
> Can the address block ...The **assignAddressField** function, implemented in [CRM/Core/BAO/UFField.php#L785](https://github.com/civicrm/civicrm-core/blob/master/CRM/Core/BAO/UFField.php#L785), is meant to return a boolean that tells if:
> Can the address block be hidden safe in the knowledge all fields are elsewhere collected (see CRM-15118)
But it only has one return statement when that covers some of the cases in which the fields can't be hidden. In other words, the function never returns true. It can let JavaScript know that all the necessary fields are present but it can't let know other PHP functions.
This issue might be related with #129.https://lab.civicrm.org/dev/financial/-/issues/177Batch Transactions: Search cannot filter by Soft-Credit2023-07-21T17:13:51ZmhowisonBatch Transactions: Search cannot filter by Soft-CreditI'm trying to create a batch of existing contacts who have all made a contribution via Donor-Advised Fund (DAF).
When I use the Find Contributions search feature, I'm able to produce a list of contacts who gave through Donor-Advised Fun...I'm trying to create a batch of existing contacts who have all made a contribution via Donor-Advised Fund (DAF).
When I use the Find Contributions search feature, I'm able to produce a list of contacts who gave through Donor-Advised Funds in FY21 (Date Received=Previous Fiscal Year, Contributions AND Their Soft Credits, Soft Credit Type=Donor-Advised Fund).
However, when I run this same query through the search feature in the 'add existing transactions to a batch' screen, the results are incorrect. It filters for previous fiscal year contributions, but it returns all contributions rather than just contributions with a soft credit type of DAF.
The Soft Credit Type criteria doesn't seem to be registering. This shows you the 'add existing transactions to a batch' screen I'm referring to: (https://docs.civicrm.org/user/en/latest/contributions/accounting-integration/#create-a-new-batch-from-existing-transactions.)https://lab.civicrm.org/dev/core/-/issues/2759Intermittent cookies error after contribution2023-03-14T07:16:59ZandyburnsIntermittent cookies error after contributionWe experience a cookies error intermittently after submitting a donation. Currently on Civi 5.35.2 and WP 5.7.2. This is multisite so may be multisite related.
![118401618_379155863073198_2489825481780434605_n](/uploads/1125c66666dd23c3...We experience a cookies error intermittently after submitting a donation. Currently on Civi 5.35.2 and WP 5.7.2. This is multisite so may be multisite related.
![118401618_379155863073198_2489825481780434605_n](/uploads/1125c66666dd23c31102ac79e053f06f/118401618_379155863073198_2489825481780434605_n.png)
A contributor does not get to the thank you page but gets the error above. The donation does succeed. This issue has persisted for a long-time (2+ years) and is a probably the worst civi issue I've encountered seeing it has to do with contribution pages and has no pattern that we can find. Tried all the go to solutions posted on SE with no luck. I would estimate that it occurs ~10-20% of the time.
This is related: https://lab.civicrm.org/dev/core/-/issues/517 and says using PHP 7.2 resolved, but were on 7.2 and now on 7.3
Some steps I've done:
- Set `Clean-up Temporary Data and Files` scheduled job run frequency from Hourly to Daily. It is on every site.
- Changed cookie domain constant in wp-config.php
```
// define('COOKIE_DOMAIN', false);
to
define('COOKIE_DOMAIN', $_SERVER\['HTTP_HOST'\] );
```
- Enabled https://lab.civicrm.org/extensions/qfsessionwarning and set timeout at 120 minutes.
`$civicrm_setting['core']['secure_cache_timeout_minutes'] = 120;`
We have considered changing how garbage collection works as sessions will die if garbage collection is run (server does that every 15 minutes).
[This post](https://civicrm.stackexchange.com/questions/39156/payments-fail-with-paypal-standard-and-could-not-find-a-valid-session-key) seems to have found a solution using the Fatal Error Handler. Is this recommended and how do you implement? Seems like a workaround symptom fix 'solution'.
Also getting this for additional event participants all the time (usually) or not at all depending on when I test: https://lab.civicrm.org/dev/core/-/issues/2708
Help?https://lab.civicrm.org/dev/financial/-/issues/181Update billing details fails when no State/Province entered2022-09-16T21:08:32ZlarsssandergreenUpdate billing details fails when no State/Province enteredUpdate billing details gives "Page could not be loaded" on submit if the user has not entered a State/Province. In some cases, it is valid for a user to not enter a State/Province if they live in a country that does not have states. This...Update billing details gives "Page could not be loaded" on submit if the user has not entered a State/Province. In some cases, it is valid for a user to not enter a State/Province if they live in a country that does not have states. This works as expected on contribution pages, etc, but not on the update billing details page. Tested on 5.35.2.
I will have a look at how this is handled for contribution pages and port the solution to update billing details, at some point.https://lab.civicrm.org/dev/financial/-/issues/182Possible regression - civicrm_line_item.line_total is incorrectly tax inclusi...2023-11-23T07:17:29ZeileenPossible regression - civicrm_line_item.line_total is incorrectly tax inclusive - potentially 5.39+In 5.39 there was a fix to stop the situation where the contribution.total_amount was recalculated on edit when tax was involved. However this fix had a bug and civicrm_line_item.line_total [stored the tax INCLUSIVE amount](see https://...In 5.39 there was a fix to stop the situation where the contribution.total_amount was recalculated on edit when tax was involved. However this fix had a bug and civicrm_line_item.line_total [stored the tax INCLUSIVE amount](see https://github.com/civicrm/civicrm-core/commit/e967ce8fe2b58b94e2163dde395542e55599da13#diff-4c9d0b1abe07057a4eea2b47bc627eecb95face8ed8d86c1c005312a52cca811L4026-L4035) We [have merged a fix to the rc](https://github.com/civicrm/civicrm-core/pull/21212) but I opened this to try to determine if any released versions are affected. This rose to a bigger issue in the rc because of changes on top of it - which assumed it was right.
The reason this went awol is largely because we had too behaviours - if the contribution bao didn't receive tax_amount it treated total_amount as exclusive and if it did it treated it as inclusive. The former was locked in by tests & not the latter leading to confusion (I'm still confused)
At this stage I'm unsure about real-world issues with this as
1) it didn't affect any core forms that I'm aware of as they pass in 'line_item' or 'skipLineItem'
2) this was the 'correct' behaviour historically for Contribution.create api calls that should have tax_amount passed in, but didn't
3) we have a [deprecation notice](https://github.com/civicrm/civicrm-core/commit/e967ce8fe2b58b94e2163dde395542e55599da13#diff-4c9d0b1abe07057a4eea2b47bc627eecb95face8ed8d86c1c005312a52cca811R193) in the code that would likely have been triggered if line item tax totals did not add up to contribution totals & it's unlikely people would miss wrong contribution totals.
However, my head breaks a bit when I try to think through this
I *think* this would pick up any wrong rows
```
SELECT
SUM(li.line_total + li.tax_amount) as lines_total,
count(li.id) as c,
SUM(c.total_amount) as total_amount
FROM civicrm_contribution c LEFT JOIN civicrm_line_item li ON li.contribution_id = c.id
GROUP BY c.id
HAVING c > 1
AND lines_total <> total_amount
```https://lab.civicrm.org/dev/core/-/issues/2796Regression? No way to change the tax rate and tax amount using buildamount hook2023-11-24T22:45:42ZPradeep Nayakpradpnayak@gmail.comRegression? No way to change the tax rate and tax amount using buildamount hookIn 5.35.x LineItem create api allowed one to set a different tax_amount in api params for a financial type that had tax enabled. The tax amount could then be calculated programatically based on region, location etc instead of using the c...In 5.35.x LineItem create api allowed one to set a different tax_amount in api params for a financial type that had tax enabled. The tax amount could then be calculated programatically based on region, location etc instead of using the config specified in the UI.
```
civicrm_api3('LineItem', 'create', [
'entity_id' => 284,
'qty' => 1,
'unit_price' => 100,
'line_total' => 100,
'entity_table' => "civicrm_contribution",
'tax_amount' => 20,
'financial_type_id' => "Donation",
'price_field_id' => "contribution_amount",
'price_field_value_id' => "single",
]);
```
Another way of calculating tax on online forms was using the [buildamount hook](https://docs.civicrm.org/dev/en/latest/hooks/hook_civicrm_buildAmount/)
```
function advanceinvoicing_civicrm_buildAmount($pageType, &$form, &$amount) {
$amount[$priceFieldId]['options'][$priceFieldValueId]['tax_rate'] = 10;
$amount[$priceFieldId]['options'][$priceFieldValueId]['tax_amount'] = 0.1;
}
```
A recent [change](https://github.com/civicrm/civicrm-core/commit/e967ce8fe2b#diff-a16d4d7449cf5f3a0616d1d282a32f27ab6d3f7d2726d076c02ad1d4d655af41R56) to the line item create function has stopped allowing tax amount to be set programmatically.
Is this intentional? Is there any way to support different tax rate for single financial type?https://lab.civicrm.org/dev/financial/-/issues/185Contributions created as pending can be later completed in test mode, behavin...2021-09-15T13:25:34ZFrancis (Agileware)Contributions created as pending can be later completed in test mode, behaving like live contributionsContributions created as pending can be later completed in test mode using the `civicrm/contribute/transact` path, behaving like live contributions.
e.g. linked Memberships are moved from Pending to New
1. As an admin, create a Contrib...Contributions created as pending can be later completed in test mode using the `civicrm/contribute/transact` path, behaving like live contributions.
e.g. linked Memberships are moved from Pending to New
1. As an admin, create a Contribution Page that performs "real-time transactions" allows the end user to leave a Contribution in Pending state (either pay later or with a payment processor that defers payment with "incomplete transaction"). For complete coverage, include membership sign up configuration.
2. As a normal user, visit the Contribution Page in live mode and submit details, but do not complete the transaction
3. As the same user, reload the Contribution Page in test mode with the relevant contribution id as URL parameter
4. Complete the transaction using test credit card details
5. As an admin, observe that the contribution has been updated as though it were completed in live mode including relevant changes to any linked entities.
Suggest that this page should probably refuse access in test mode if it attempts to load an existing contribution that is not marked `is_test`https://lab.civicrm.org/dev/financial/-/issues/186Accounting entries incorrect in a number of cases... especially with pending ...2023-05-01T17:18:33ZJamie Novick - CompucoAccounting entries incorrect in a number of cases... especially with pending refunds and overpaymentsFirstly - sorry for the wall of text. This has ended up being a much longer piece of analysis than expected but I thought I should share it so that we can agree a way ahead.
This all started as we were looking at the best way to impleme...Firstly - sorry for the wall of text. This has ended up being a much longer piece of analysis than expected but I thought I should share it so that we can agree a way ahead.
This all started as we were looking at the best way to implement refunds for a payment processor. Some work was done here to discuss this: https://lab.civicrm.org/dev/financial/-/issues/87 but I can see this stalled somewhat - probably due to some of the issues I'm going to discuss here.
# Background: The problems with the contribution status field
Just for complete disclosure I hate the CiviCRM contribution status field. Hopefully everyone already knows that. I've probably started discussions on getting rid of it at every CiviCON and documented this on every financial gitlab ticket. It is the biggest blocker we have to reaching an accounting implementation that aligns to standard concepts of accounting.
Why is that?
We appear to have 2 use cases for it, both of which are undocumented.
1. For users to quickly change whether the amounts owed are
a) expected to be paid or
b) are paid.
2. To reflect the payment status of the contribution
i.e. whether the total sum of the line items is >, < or = to the net amount paid in or refunded on that contribution (this is important and I’ll come back to it)
This is a bit of a problem as it’s all a bit chicken and egg! If the user changes the status we need to understand whether we should be creating payments or not (or creating refund payments or not), and if the user records payments or refund payments we need to update the status to something meaningful and consistent.
The following is an analysis of the allowed contribution status changes from and to that a user can perform:
**Table 1:**
- Black = Option hidden
- X - Val = Option shown but user cannot change to it as we have a validation message
- Y (black) = Option shown, user can select this and I don’t see issues with accounting that occurs
- N/A = Couldn’t test
- Y (red) = Option shown, user can select this and there are issues with accounting that occurs
![Screenshot_2021-09-15_at_15.33.12](/uploads/8db825597f091f0bf34d590b38b2ed2f/Screenshot_2021-09-15_at_15.33.12.png)
I’d note that it’s a bit confusing to users to see a value in the dropdown they can’t use. I think we should just hide values they cannot select as this would be much clearer for them.
We end up with some very strange and in some cases incorrect behaviour when allowing users to change the status in a "partially paid" and "pending refund" scenarios:
**Table 2:**
![Screenshot_2021-09-15_at_15.47.11](/uploads/827aafd531d7dbe54866b8237758ce42/Screenshot_2021-09-15_at_15.47.11.png)
# Ground Rules
As such I would like to suggest some ground rules to work towards:
1. The only time a user can change the status of a contribution should be when it has no payments or refunds attached to it.
2. Once a payment has been received we manage everything through either the line item editor or the “change selections” on the event form or "record payment” or “record refund”. If any changes to each are made to any of these we recalculate the contribution status based on one set of rules:
The rules to calculate the contribution status should be:
**Table 3:**
![Screenshot_2021-09-15_at_15.38.43](/uploads/b77e518446862474f4e6823e045e11a3/Screenshot_2021-09-15_at_15.38.43.png)
Delving deeper, there are a number of situations within CiviCRM that do not confirm to this currently and instead we end up with a status this is confusing for the user:
(Note this is not be a comprehensive list - just to give some examples).
**Table 4:**
![Screenshot_2021-09-15_at_15.39.45](/uploads/3a9e238f9ec1071de6b056c543c68522/Screenshot_2021-09-15_at_15.39.45.png)
# Specific Scenarios
This all said there are some specific use cases that we need to consider and have appropriate workflows for:
Table 5:![Screenshot_2021-09-15_at_15.40.49](/uploads/a922e2c0f5a52b92adba895c5fd89654/Screenshot_2021-09-15_at_15.40.49.png)
# Actions:
As such I think the actions could be:
1. Agree to the "ground rules" for the contribution status field above so we are all working towards a common goal.
1. I think we should hide values they cannot select as this would be much clearer for them. i.e. all items that are marked as “X-val” in the table above. Perhaps we can refactor the code that show/hides the options to a central place.
1. I think we should change the way the status field is calculated to use the business logic suggested in Table 3 above in all cases. This shouldn’t be something that extensions should take care of but should be calculated any time a contribution haas it’s line items / financial transactions changed. Obviously I don’t know what this means from a code perspective and I assume this would be a significant refactor but I think we could clear up a lot of business logic by doing so.
1. Discuss and agree which of the options from table 5.1 are suitable and then agree the UX for this.
1. So that we can then remove the option to make the status changes as per table 1: 3f, 4e, 4f without degrading users ability to do the things they need.
I actually think if we do these things we can consider CiviCRM’s accounting to be “correct” if not completely intuitive (at least for now).
@mattwire @eileen @KarinG @JoeMurrayhttps://lab.civicrm.org/dev/core/-/issues/2846Improve validation of start and end dates for events, contribution pages and ...2023-11-23T07:09:32ZJKingsnorthImprove validation of start and end dates for events, contribution pages and more**Motivation**
There are holes in the validation of start and end dates on contribution pages.
eg:
- Create a new contribution page
- Enter a start date
- Enter an end 'time' but leave the date part empty:
![image](/uploads/e75411e618...**Motivation**
There are holes in the validation of start and end dates on contribution pages.
eg:
- Create a new contribution page
- Enter a start date
- Enter an end 'time' but leave the date part empty:
![image](/uploads/e75411e61813ee5ae55b808f49fa3637/image.png)
- Result is a fatal DB error.
This may seem like an edge-case, but it can also happen when you are 'scrolling' down the page, and accidentally 'scroll' on the time field. This sets a time.
The same is true for event pages event 'start' and 'end' dates.
The same is true for event registration open and close dates:
![image](/uploads/270044386b6b0895da079268d9147e8c/image.png)
The same is true for price fields, with the 'active on' and 'expire on', where setting a time without a date results in a fatal error and the endlessly spinning civi icon of doom.
And for price fields, there is currently no validation to ensure the 'expire on' is before the 'active on'.
Creating campaigns also:
![image](/uploads/c51922aa11942eafa7e58e6d153a5fae/image.png)
**Proposed solution**
I think the validation should really be in the date picker element somewhere, to ensure a date and time are both set - where we expect it.
But I also thought we could take this opportunity to standardise the validation of start and end dates in:
- Event pages (event start and end)
- Event pages (registration open/close)
- Contribution pages (start and end)
- Price fields (active on, expire on)
- Campaign (start, end dates)
I am working on this now.
**After**
![image](/uploads/968b1bafab430a8c7cbc2e5a684e1b01/image.png)
**Follow-up / part 2**
Validation at the API/BAO level?https://lab.civicrm.org/dev/financial/-/issues/188Financial Items incorrectly recorded when using Payment API2021-10-27T16:53:49ZhaystackFinancial Items incorrectly recorded when using Payment APIFinally found the time to return to the Order & Payment API and test the changes that @eileen and @KarinG have worked on. So much better now that I can update a contribution without the tax amounts being recalculated!
Warning, this is g...Finally found the time to return to the Order & Payment API and test the changes that @eileen and @KarinG have worked on. So much better now that I can update a contribution without the tax amounts being recalculated!
Warning, this is going to be a long issue because it seems to involve two (maybe overlapping) issues which I haven't been able to separate. So... the scenario that I'm testing is where:
* I create multiple Line Items in an Order that are a mix of taxable and non-taxable.
* They are also a mix of Participant and Membership Line Items.
* The Line Items can be in any sequence when the Order is created.
To tease out what's going on, I tested the following:
1. Taxable Participant ($25) followed by Non-taxable Membership ($50)
2. Non-taxable Membership ($50) followed by Taxable Participant ($25)
3. Taxable Participant ($50) followed by Non-taxable Membership ($50)
4. Non-taxable Membership ($50) followed by Taxable Participant ($50)
These happen to be for Events ("Summer Solstice Festival Day Concert" and "Fall Fundraiser Dinner") and Membership Types (Student) in the CiviCRM Sample Data - though if you want to use those to replicate then it should be noted that "Summer Solstice Festival Day Concert" is incorrectly set to the "Member Dues" Financial Type. @kcristiano was invaluable in setting up CiviCRM's Sample Data with correct Financial Types and I assume there's nothing amiss in that regard.
## Taxable Participant ($25) followed by Non-taxable Membership ($50)
The procedure is commented upon in full for this first test. I'll just post the annotated logs for the subsequent tests since the procedure is identical.
![civicrm-E25-M50-order](/uploads/74c5bdb91fb8ee46d73a0f525a7552ad/civicrm-E25-M50-order.png)
The params for Order.create are as follows:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 11:28:21
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 79.84 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Solstice Ticket x 1, Student Membership x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[source] => Shop: Solstice Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[label] => Solstice Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
[18] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
)
)
```
The CiviCRM API (yeah, I know I should convert to API4 but that's another story) returns:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011112821
[non_deductible_amount] =>
[total_amount] => 79.844775 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 79.844775 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 25.00
[label] => Solstice Ticket
[line_total] => 25.00
[tax_amount] => 4.844775 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
)
)
)
)
```
So far so good - though it seems odd that the Total Amount, Net Amount and Tax Amount are reported as having 6 decimal places given that a pre-calculated Tax Amount to 2 decimal places was passed in.
Let's have a look at the Bookkeeping Report at this stage:
![civicrm-E25-M50-pre](/uploads/0fa2ceec4ca463ce50a38bb0ff8b96ae/civicrm-E25-M50-pre.png)
Looks good. The database looks good too:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:28:21 | 79.84 | 79.84 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 25.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 4.84 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 50.00 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Submit the `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 79.84
[trxn_date] => 2021-10-11 11:30:58
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
And the logs show:
```
PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
The API result is:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 11:30:58
[total_amount] => 79.84
[fee_amount] => 0.00
[net_amount] => 79.84
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Nice. So what's in the database?
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Solstice Ticket | 25.00 | USD | 15 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 11:28:23 | 2021-10-11 11:28:21 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:28:21 | 79.84 | 79.84 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 11:30:58 | 79.84 | 79.84 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 |
| 204 | civicrm_financial_item | 104 | 101 | 25.00 |
| 205 | civicrm_financial_item | 105 | 101 | 4.84 |
| 206 | civicrm_financial_item | 106 | 101 | 50.00 |
| 207 | civicrm_contribution | 103 | 102 | 79.84 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 25.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 105 | 102 | 4.84 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 105 | 102 | 4.84 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
The Bookkeeping Report seems to reflect the odd assignment of Financial Items:
![civicrm-E25-M50-post](/uploads/2e550b6a4893a9d8960ec4b0a2bcff14/civicrm-E25-M50-post.png)
As you can see, it lists all of the `1100` entries as "Event Fee Taxable". The "Member Dues" item seems to have been switched to "Event Fee Taxable". There also seems to be a duplicate Tax Amount entry. This may be important below when the Financial Item amounts are identical.
## Non-taxable Membership ($50) followed by Taxable Participant ($25)
Sequence of Line Items is reversed:
![civicrm-M50-E25-order](/uploads/3fb49ce43318c51ba2da2f6cd8be3470/civicrm-M50-E25-order.png)
Order API params:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 11:36:49
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 79.84 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Student Membership x 1, Solstice Ticket x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
[18] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[source] => Shop: Solstice Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[label] => Solstice Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
)
)
```
API returns:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011113649
[non_deductible_amount] =>
[total_amount] => 79.844775 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 79.844775 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 4.84 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 25.00
[label] => Solstice Ticket
[line_total] => 25.00
[tax_amount] => 4.844775 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
)
)
)
)
```
Database looks good:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:36:49 | 79.84 | 79.84 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 50.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 25.00 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 4.84 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Bookkeeping confirms this:
![civicrm-M50-E25-pre](/uploads/87857b96e6fb0b8b2379c29e832da746/civicrm-M50-E25-pre.png)
Submit the `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 79.84
[trxn_date] => 2021-10-11 11:38:59
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
Logs again show:
```
[11-Oct-2021 10:39:00 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
[11-Oct-2021 10:39:01 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
API result:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 11:38:59
[total_amount] => 79.84
[fee_amount] => 0.00
[net_amount] => 79.84
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Database again shows the wrongly assigned Financial Items:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 79.84 | 0.00 | 79.84 | 4.84 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 25.00 | 25.00 | 5 | 4.84 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Student Membership | 50.00 | USD | 2 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Solstice Ticket | 25.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 11:36:51 | 2021-10-11 11:36:49 | 210 | Sales Tax | 4.84 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 11:36:49 | 79.84 | 79.84 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 11:38:59 | 79.84 | 79.84 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 79.84 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 25.00 |
| 206 | civicrm_financial_item | 106 | 101 | 4.84 |
| 207 | civicrm_contribution | 103 | 102 | 79.84 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 106 | 102 | 4.84 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 25.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 106 | 102 | 4.84 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
Again, there also seems to be a duplicate Tax Amount entry.
Bookkeeping Report confirmation of the odd assignment:
![civicrm-M50-E25-post](/uploads/7124d730e050f3f288de87ed29bfeb1b/civicrm-M50-E25-post.png)
The "Event Fee Taxable" item seems to have been switched to "Member Dues".
## Taxable Participant ($50) followed by Non-taxable Membership ($50)
Sequence as per first test:
![civicrm-E50-M50-order](/uploads/3cadc61cf6ecceeb570c7298ae0aafb5/civicrm-E50-M50-order.png)
Order API params:
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 1 <-- Non-taxable Financial Type
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[receive_date] => 2021-10-11 12:01:02
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 109.69 <-- Note: Total Amount has 2 decimal places
[source] => Shop
[campaign_id] => 3
[note] => Fundraiser Dinner Ticket x 1, Student Membership x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 1
[contact_id] => 210
[role_id] => 1
[source] => Shop: Fundraiser Dinner Ticket
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 9.69 <-- Note: Tax Amount has 2 decimal places
[label] => Fundraiser Dinner Ticket
[entity_table] => civicrm_participant
[financial_type_id] => 5 <-- Taxable Financial Type
)
)
)
[18] => Array
(
[params] => Array
(
[membership_type_id] => 2
[source] => Shop
[contact_id] => 210
[skipStatusCal] => 1
[status_id] => Pending
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 1
[unit_price] => 50.00
[qty] => 1
[line_total] => 50.00
[tax_amount] => 0.00
[label] => Student Membership
[entity_table] => civicrm_membership
[financial_type_id] => 2 <-- Non-taxable Financial Type
[membership_type_id] => 2
)
)
)
)
)
```
API return:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 103
[values] => Array
(
[103] => Array
(
[id] => 103
[contact_id] => 210
[financial_type_id] => 1
[contribution_page_id] =>
[payment_instrument_id] => 4
[receive_date] => 20211011120102
[non_deductible_amount] =>
[total_amount] => 109.68955 <-- Note: Total has 6 decimal places
[fee_amount] => 0
[net_amount] => 109.68955 <-- Note: Net Amount has 6 decimal places
[trxn_id] => WooCommerce Order - 2247
[invoice_id] => 2247_woocommerce
[invoice_number] => INV_103
[currency] => USD
[cancel_date] =>
[cancel_reason] =>
[receipt_date] =>
[thankyou_date] =>
[source] => Shop
[amount_level] =>
[contribution_recur_id] =>
[is_test] =>
[is_pay_later] => 1
[contribution_status_id] => 2
[address_id] =>
[check_number] =>
[campaign_id] => 3
[creditnote_id] =>
[tax_amount] => 9.69 <-- Note: Tax Amount has 2 decimal places
[revenue_recognition_date] =>
[is_template] =>
[contribution_type_id] => 1
[line_item] => Array
(
[0] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_participant
[unit_price] => 50.00
[label] => Fundraiser Dinner Ticket
[line_total] => 50.00
[tax_amount] => 9.68955 <-- Note: Tax Amount has 6 decimal places
[financial_type_id] => 5
[entity_id] => 56
)
[1] => Array
(
[qty] => 1
[price_field_id] => 1
[price_field_value_id] => 1
[entity_table] => civicrm_membership
[unit_price] => 50.00
[label] => Student Membership
[line_total] => 50.00
[tax_amount] => 0
[financial_type_id] => 2
[membership_type_id] => 2
[entity_id] => 35
)
)
)
)
)
```
Again, database looks good:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 3 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:01:02 | 109.69 | 109.69 | 0 | 2 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 | <-- Correct Entity ID.
| 204 | civicrm_financial_item | 104 | 101 | 50.00 | <-- Correct Entity ID.
| 205 | civicrm_financial_item | 105 | 101 | 9.69 | <-- Correct Entity ID.
| 206 | civicrm_financial_item | 106 | 101 | 50.00 | <-- Correct Entity ID.
+-----+------------------------+-----------+-------------------+--------+
```
Bookkeeping Report confirms:
![civicrm-E50-M50-pre](/uploads/728d5845348f0f8bf289f5a7fbcb84f6/civicrm-E50-M50-pre.png)
This is where things get a bit weird(er).
Call `Payment.create` with:
```
[params] => Array
(
[contribution_id] => 103
[total_amount] => 109.69
[trxn_date] => 2021-10-11 12:04:49
[trxn_id] => WooCommerce Order - 2247
[payment_instrument_id] => 4
)
```
Logs (as usual) show:
```
[11-Oct-2021 11:04:50 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
[11-Oct-2021 11:04:51 UTC] PHP Notice: Undefined variable: CRM16923AnUnreliableMethodHasBeenUserToDeterminePaymentProcessorFromEvent in /Users/interactivist/Sites/civicrm/civicrm.events.tec.latest/httpdocs/wp-content/plugins/civicrm/civicrm/CRM/Contribute/BAO/Contribution.php on line 2724
```
API result is:
```
[result] => Array
(
[is_error] => 0
[version] => 3
[count] => 1
[id] => 102
[values] => Array
(
[102] => Array
(
[id] => 102
[from_financial_account_id] => 7
[to_financial_account_id] => 6
[trxn_date] => 2021-10-11 12:04:49
[total_amount] => 109.69
[fee_amount] => 0.00
[net_amount] => 109.69
[currency] => USD
[is_payment] => 1
[trxn_id] => WooCommerce Order - 2247
[trxn_result_code] =>
[status_id] => 1
[payment_processor_id] =>
)
)
)
```
Database has the same pattern of mis-assignment:
```
SELECT id, total_amount, fee_amount, net_amount, tax_amount FROM `civicrm_contribution` WHERE id = '103';
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
| 108 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 107 |
| 106 | 2021-10-11 12:01:04 | 2021-10-11 12:01:02 | 210 | Student Membership | 50.00 | USD | 2 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:01:02 | 109.69 | 109.69 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 12:04:49 | 109.69 | 109.69 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 9.69 |
| 206 | civicrm_financial_item | 106 | 101 | 50.00 |
| 207 | civicrm_contribution | 103 | 102 | 109.69 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 105 | 102 | 9.69 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 105 | 102 | 9.69 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
However, this time the consequences of this combination of mismatch and duplicate entry gives a different result in the Bookkeeping Report:
![civicrm-E50-M50-post](/uploads/afb612896bb51e3db2afd7b67b176c9d/civicrm-E50-M50-post.png)
This time, it seems, there are only 5 Financial Items visible.
## Non-taxable Membership ($50) followed by Taxable Participant ($50)
![civicrm-M50-E50-order](/uploads/0f34a882ab9bfb12322254183951b89a/civicrm-M50-E50-order.png)
I'll skip the API calls for this one - if you want them, I'm happy to provide.
The `Order.create` process goes fine, but we're left with the following in the database:
```
+-----+--------------+------------+------------+------------+
| id | total_amount | fee_amount | net_amount | tax_amount |
+-----+--------------+------------+------------+------------+
| 103 | 109.69 | 0.00 | 109.69 | 9.69 |
+-----+--------------+------------+------------+------------+
SELECT id, entity_table, qty, unit_price, line_total, financial_type_id, tax_amount FROM `civicrm_line_item` WHERE contribution_id = '103';
+-----+---------------------+------+------------+------------+-------------------+------------+
| id | entity_table | qty | unit_price | line_total | financial_type_id | tax_amount |
+-----+---------------------+------+------------+------------+-------------------+------------+
| 107 | civicrm_membership | 1.00 | 50.00 | 50.00 | 2 | 0.00 |
| 108 | civicrm_participant | 1.00 | 50.00 | 50.00 | 5 | 9.69 |
+-----+---------------------+------+------------+------------+-------------------+------------+
SELECT * FROM `civicrm_financial_item` WHERE contact_id = '210';
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| id | created_date | transaction_date | contact_id | description | amount | currency | financial_account_id | status_id | entity_table | entity_id |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
| 104 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Student Membership | 50.00 | USD | 2 | 1 | civicrm_line_item | 107 |
| 105 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Fundraiser Dinner Ticket | 50.00 | USD | 15 | 3 | civicrm_line_item | 108 |
| 106 | 2021-10-11 12:21:33 | 2021-10-11 12:21:31 | 210 | Sales Tax | 9.69 | USD | 17 | 3 | civicrm_line_item | 108 |
+-----+---------------------+---------------------+------------+--------------------------+--------+----------+----------------------+-----------+-------------------+-----------+
SELECT id, from_financial_account_id, to_financial_account_id, trxn_date, total_amount, net_amount, is_payment, status_id, payment_instrument_id FROM `civicrm_financial_trxn` WHERE id > '100';
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| id | from_financial_account_id | to_financial_account_id | trxn_date | total_amount | net_amount | is_payment | status_id | payment_instrument_id |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
| 101 | NULL | 7 | 2021-10-11 12:21:31 | 109.69 | 109.69 | 0 | 2 | 4 |
| 102 | 7 | 6 | 2021-10-11 12:25:19 | 109.69 | 109.69 | 1 | 1 | 4 |
+-----+---------------------------+-------------------------+---------------------+--------------+------------+------------+-----------+-----------------------+
SELECT * FROM `civicrm_entity_financial_trxn` WHERE financial_trxn_id > '100';
+-----+------------------------+-----------+-------------------+--------+
| id | entity_table | entity_id | financial_trxn_id | amount |
+-----+------------------------+-----------+-------------------+--------+
| 203 | civicrm_contribution | 103 | 101 | 109.69 |
| 204 | civicrm_financial_item | 104 | 101 | 50.00 |
| 205 | civicrm_financial_item | 105 | 101 | 50.00 |
| 206 | civicrm_financial_item | 106 | 101 | 9.69 |
| 207 | civicrm_contribution | 103 | 102 | 109.69 | <-- Correct Entity ID.
| 208 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Correct Entity ID.
| 209 | civicrm_financial_item | 106 | 102 | 9.69 | <-- Duplicated in 211?
| 210 | civicrm_financial_item | 104 | 102 | 50.00 | <-- Wrong Entity ID.
| 211 | civicrm_financial_item | 106 | 102 | 9.69 | <-- Duplicate of 209?
+-----+------------------------+-----------+-------------------+--------+
```
The Bookkeeping Report shows the end result:
![civicrm-M50-E50-post](/uploads/4cefa4c34c1b02956fdc9ac3e62c8d4c/civicrm-M50-E50-post.png)
## Final thought
My best guess at the moment is that the combination of the mis-assigned `entity_id`s for the Financial Items _plus_ the effect of the extra entry in the `civicrm_entity_financial_trxn` table leads to the mix of 5 or 6 Items visible in the Bookkeeping Report.
If you've got to here, thanks for reading! Hope this helps fix what seemed to be mind-bending symptoms.https://lab.civicrm.org/dev/financial/-/issues/189Creating an Order fails with `Line item total doesn't match total amount` due...2023-06-19T09:18:26ZhaystackCreating an Order fails with `Line item total doesn't match total amount` due to the mismatch of 2 vs 6 decimal placesOpening this instead of conflating two issues in #188 - specifically referring to [this thread](https://lab.civicrm.org/dev/financial/-/issues/188#note_66286). As the subject says, creating an Order can fail with the error `Line item tot...Opening this instead of conflating two issues in #188 - specifically referring to [this thread](https://lab.civicrm.org/dev/financial/-/issues/188#note_66286). As the subject says, creating an Order can fail with the error `Line item total doesn't match total amount` because of the mismatch of 2 vs 6 decimal places when calculating the Order total from the amounts and tax amounts in the Line Items.
Say, for example, I have two Taxable Line Items in an Order both of which have had their Tax Amount pre-calculated - in this case by WooCommerce but I assume this could equally be the result from any external Payment Processor. All the amounts in the following params are correct as far as WooCommerce is concerned. Both CiviCRM and WooCommerce have `19.37910000` configured as the relevant Tax Rate.
```
[params] => Array
(
[contact_id] => 210
[financial_type_id] => 5
[payment_instrument_id] => 4
[trxn_id] => WooCommerce Order - 2250
[invoice_id] => 2250_woocommerce
[receive_date] => 2021-10-13 19:34:30
[contribution_status_id] => Pending
[is_pay_later] => 1
[total_amount] => 77.59 <-- The pre-calculated Total Amount
[tax_amount] => 12.59 <-- The pre-calculated Total Tax Amount
[source] => Shop
[campaign_id] => 3
[note] => Solstice Ticket (Bass) x 1, Solstice Ticket (Tenor) x 1
[line_items] => Array
(
[17] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[price_set_id] => 8
[fee_level] => Bass
[fee_amount] => 29.84
[source] => Shop: Solstice Ticket (Bass)
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 9
[unit_price] => 25.00
[qty] => 1
[line_total] => 25.00 <-- The Line Total
[tax_amount] => 4.84 <-- The pre-calculated Tax Amount
[label] => Bass
[entity_table] => civicrm_participant
[financial_type_id] => 5
[price_field_value_id] => 19
)
)
)
[18] => Array
(
[params] => Array
(
[event_id] => 2
[contact_id] => 210
[role_id] => 1
[price_set_id] => 8
[fee_level] => Tenor
[fee_amount] => 47.75
[source] => Shop: Solstice Ticket (Tenor)
[status_id] => Pending from pay later
)
[line_item] => Array
(
[0] => Array
(
[price_field_id] => 9
[unit_price] => 40.00
[qty] => 1
[line_total] => 40.00 <-- The Line Total
[tax_amount] => 7.75 <-- The pre-calculated Tax Amount
[label] => Tenor
[entity_table] => civicrm_participant
[financial_type_id] => 5
[price_field_value_id] => 20
)
)
)
)
)
```
The Order API recalculates `$order->getTotalAmount()` and [then compares it with what's passed in via the API](https://github.com/civicrm/civicrm-core/blob/f73d3ac9a9979d8a616b0e2df53a459f7e6a84d7/api/v3/Order.php#L113). However, the rounding to 6 decimal places (rather than 2) triggers `CRM_Contribute_Exception_CheckLineItemsException` because `$params['total_amount'] = 77.59` does not equal `$order->getTotalAmount() = 77.596415`.
WooCommerce appears to _round_ the Line Items before _summing_ them, so:
* Line Item 1: `1.193791 * 25 = 29.844775 => 29.84`
* Line Item 2: `1.193791 * 40 = 47.75164 => 47.75`
* Total of rounded Line Items: `77.59`
CiviCRM, appears to _sum_ the Line Items before _rounding_ them, so:
* Line Item 1: `1.193791 * 25 = 29.844775`
* Line Item 2: `1.193791 * 40 = 47.75164`
* Total of summed Line Items: `77.596415`
`CRM_Utils_Money::equals()` then rounds _up_ not _down_ because of the extra precision and :boom: the above params cannot successfully create an Order.
My suggestion is that CiviCRM should:
* Verify the amount in each Line Item to the number of decimal places for the given currency.
* Keep a running total of those amounts.
* Compare the final running total with the `total_amount`.https://lab.civicrm.org/dev/core/-/issues/2987API call Contribution.repeattransaction fails with recurring contributions th...2023-03-20T01:51:15ZandrewcormickdockeryAPI call Contribution.repeattransaction fails with recurring contributions that have a one-to-one custom group record attachedOverview
----------------------------------------
Re: https://chat.civicrm.org/civicrm/pl/i8fht46isjn5uj4dbdosrqpqmr
The API call Contribution.repeattransaction appears to write a custom group record twice when completing a transaction....Overview
----------------------------------------
Re: https://chat.civicrm.org/civicrm/pl/i8fht46isjn5uj4dbdosrqpqmr
The API call Contribution.repeattransaction appears to write a custom group record twice when completing a transaction. On the second attempt, a database error caused by the clashing unique key is generated. The transaction appears to end up being recorded correctly, however the fatal error means that processes which call this function repeatedly end up not completing. Our use case involves finding all Eway transactions whose next scheduled contribution date is today, and then calling Contribution.repeattransaction for each one.
Reproduction steps
----------------------------------------
1. Ensure a one-to-one custom group exists against contributions.
1. Ensure a recurring contribution exists with a record in that custom group against the transaction.
1. Run `drush cvapi Contribution.repeattransaction contribution_status_id=Completed total_amount=<some value> contribution_recur_id=<contribution_recur_id in previous step> trxn_id=<random unique number>`
Current behaviour
----------------------------------------
Failure message appears: `DB Error: already exists`; process terminates. Record appears to be correctly produced in CiviCRM.
Log details:
```
Dec 07 12:40:11 [error] $Fatal Error Details = Array
(
[callback] => Array
(
[0] => CRM_Core_Error
[1] => exceptionHandler
)
[code] => -5
[message] => DB Error: already exists
[mode] => 16
[debug_info] => INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']
[type] => DB_Error
[user_info] => INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']
[to_string] => [db_error: message="DB Error: already exists" code=-5 mode=callback callback=CRM_Core_Error::exceptionHandler prefix="" info="INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_field_2`,`custom_field_3`,`custom_field_4`,`entity_id` ) VALUES ( 1,'Some Text','','22943426',123456 ) [nativecode=1062 ** Duplicate entry '123456' for key 'unique_entity_id']"]
)
Dec 07 12:40:11 [debug] $backTrace = #0 /path/to/civi/root/civicrm/CRM/Core/Error.php(942): CRM_Core_Error::backtrace("backTrace", TRUE)
#1 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(922): CRM_Core_Error::exceptionHandler(Object(DB_Error))
#2 /path/to/civi/root/civicrm/vendor/pear/db/DB.php(997): PEAR_Error->__construct("DB Error: already exists", -5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `hide_from_slider_182`,`custom_...")
#3 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(575): DB_Error->__construct(-5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#4 /path/to/civi/root/civicrm/vendor/pear/pear-core-minimal/src/PEAR.php(223): PEAR->_raiseError(Object(DB_mysqli), NULL, -5, 16, (Array:2), "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", "DB_Error", TRUE)
#5 /path/to/civi/root/civicrm/vendor/pear/db/DB/common.php(1928): PEAR->__call("raiseError", (Array:7))
#6 /path/to/civi/root/civicrm/vendor/pear/db/DB/mysqli.php(936): DB_common->raiseError(-5, NULL, NULL, "INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", "1062 ** Duplicate entry '123456' for key 'unique_entity_id'")
#7 /path/to/civi/root/civicrm/vendor/pear/db/DB/mysqli.php(406): DB_mysqli->mysqliRaiseError()
#8 /path/to/civi/root/civicrm/vendor/pear/db/DB/common.php(1234): DB_mysqli->simpleQuery("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#9 /path/to/civi/root/civicrm/packages/DB/DataObject.php(2696): DB_common->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#10 /path/to/civi/root/civicrm/packages/DB/DataObject.php(1829): DB_DataObject->_query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#11 /path/to/civi/root/civicrm/CRM/Core/DAO.php(468): DB_DataObject->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...")
#12 /path/to/civi/root/civicrm/CRM/Core/DAO.php(1621): CRM_Core_DAO->query("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", TRUE)
#13 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(271): CRM_Core_DAO::executeQuery("INSERT INTO civicrm_value_contribution_custom_group_1 ( `custom_field_1`,`custom_...", (Array:5))
#14 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(391): CRM_Core_BAO_CustomValueTable::create((Array:2), "create")
#15 /path/to/civi/root/civicrm/CRM/Core/BAO/CustomValueTable.php(411): CRM_Core_BAO_CustomValueTable::store((Array:5), "civicrm_contribution", 123456, "create")
#16 /path/to/civi/root/civicrm/CRM/Core/DAO.php(2022): CRM_Core_BAO_CustomValueTable::postProcess((Array:5), "civicrm_contribution", 123456, "Contribution", "create")
#17 /path/to/civi/root/civicrm/CRM/Contribute/BAO/Contribution.php(2394): CRM_Core_DAO->copyCustomFields(123455, 123456)
#18 /path/to/civi/root/civicrm/CRM/Contribute/BAO/Contribution.php(4000): CRM_Contribute_BAO_Contribution::repeatTransaction((Array:10), (Array:18))
#19 /path/to/civi/root/civicrm/api/v3/Contribution.php(687): CRM_Contribute_BAO_Contribution::completeOrder((Array:10), "2269", NULL, NULL)
#20 /path/to/civi/root/civicrm/api/v3/Contribution.php(635): _ipn_process_transaction((Array:7), Object(CRM_Contribute_BAO_Contribution), (Array:10), (Array:10))
#21 /path/to/civi/root/civicrm/Civi/API/Provider/MagicFunctionProvider.php(89): civicrm_api3_contribution_repeattransaction((Array:7))
#22 /path/to/civi/root/civicrm/Civi/API/Kernel.php(149): Civi\API\Provider\MagicFunctionProvider->invoke((Array:8))
#23 /path/to/civi/root/civicrm/Civi/API/Kernel.php(81): Civi\API\Kernel->runRequest((Array:8))
#24 /path/to/civi/root/civicrm/api/api.php(132): Civi\API\Kernel->runSafe("contribution", "repeattransaction", (Array:5))
#25 /path/to/civi/root/civicrm/drupal/drush/civicrm.drush.inc(1581): civicrm_api3("Contribution", "repeattransaction", Array:5))
#26 /usr/local/src/drush/includes/command.inc(422): drush_civicrm_api("Contribution.repeattransaction", "contribution_status_id=Completed", "total_amount=11.00", "contribution_recur_id=22663", "trxn_id=aa11bb22cc33dd47")
#27 /usr/local/src/drush/includes/command.inc(231): _drush_invoke_hooks((Array:38), (Array:5))
#28 /usr/local/src/drush/includes/command.inc(199): drush_command("Contribution.repeattransaction", "contribution_status_id=Completed", "total_amount=11.00", "contribution_recur_id=22663", "trxn_id=aa11bb22cc33dd47")
#29 /usr/local/src/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch((Array:38))
#30 /usr/local/src/drush/includes/preflight.inc(67): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#31 /usr/local/src/drush/drush.php(12): drush_main()
#32 {main}
```
Expected behaviour
----------------------------------------
Fatal error not raised; no attempt to write the exact same record twice.
I have mitigated this behaviour with the following patch, but this is clearly a temporary solution and the real root cause needs to be found:
```
diff --git a/CRM/Core/BAO/CustomValueTable.php b/CRM/Core/BAO/CustomValueTable.php
index 8556c4ec21..6773b994c4 100644
--- a/CRM/Core/BAO/CustomValueTable.php
+++ b/CRM/Core/BAO/CustomValueTable.php
@@ -55,7 +55,7 @@ class CRM_Core_BAO_CustomValueTable {
$hookOP = 'edit';
}
else {
- $sqlOP = "INSERT INTO $tableName ";
+ $sqlOP = "INSERT IGNORE INTO $tableName ";
$where = NULL;
$hookOP = 'create';
}
```
Environment information
----------------------------------------
* __Browser:__ _Firefox 94.0.2_
* __CiviCRM:__ _5.43.2_
* __PHP:__ _7.3_
* __CMS:__ _Drupal 7.82_
* __Database:__ _MariaDB 10.4.21_
* __Web Server:__ _Nginx 1.10.3_