Contribution status’s are ambiguous (WIP)
Background:
Contribution status’s are ambiguous in CiviCRM and can lead to confusion over the current state of an invoice or contribution and whether there are amounts owed or owing to the contributor. This leads to a wealth of problems about what the intention of the user really is in a number of scenarios when they change the contribution status.
Examples:
| Example | Narrative | Video |
|---|---|---|
| Example 1a: Overpayment / pending refund | A contribution which is created for $100 donation, but pending. We then record an overpayment of $120. Hypothetically the contribution is overpaid by $20 and should be pending refund. This is not shown in CiviCRM. There is nothing showing the user that there is an overpayment of $20. | 1-Overpayment |
| Example 1b: Change contribution status to refunded | We can further demonstrate this by taking the same contribution and then changing the status of the contribution to refunded. This raises the question of what this means? Does this reflect the users intention to: Cancel the original debt of $100 (i.e. to credit note the original amounts owed and clear the balance Or reflect the fact a refund was made for the payment of $120, but that they are still owed the original $100 Or both? CiviCRM has refunded the original value of the invoice, i.e. $100, even though the payment of $120 was received. As such we still owe $20 but this is not shown in CiviCRM. | 2-Change_status_to_refunded |
| Example 2: A typo entered in the wrong contribution amount | Add a contribution for $100 and set as completed. Realised that actually the contribution was $120. System will automatically add the additional $20 payment. Note: If this was a change in the line items of an invoice, it would not be safe to assume we received the additional payment. The aim of the user might be to state the obligation to pay was $120. Also it probably wasn’t 2 separate payments and hence we should reverse out the first payment (-100) and then make a second payment for the full amount (+120) | Updating_a_contribution_amount_error |
Much of this stems from the contribution status field being both be the driver of changes to the contribution (i.e. by changing the status to completed or refunded) or changed itself (i.e. the result of changes to the position of the contribution) by recording payments or refunds.
As such this is a proposal to simplify this workflow so that the the contribution status is only the result of the status of the contribution, and can no longer be used to change the status of the contribution.
We would retain the ability to make changes to the contribution status via the API for backwards compatibility until extensions can adopt the new model.
Further ramblings (feel free to skip this part if you are not too interested in accounting principles)
Accounting has simple and sounds principles for dealing with all of this.
“Obligations” are managed by 2 documents: Invoices and Credit notes. An invoice would stipulate what is obligated. This can be reduced by creating a credit note, which would then reduce the obligated amount.
It’s worth noting that for sales tax (for example VAT) purposes invoices should be sequentially numbered and should not be edited once they are generated. Hence usually you would not just simply delete an invoice to reduce the obligated amount, you would create a credit note for some or all of the amount, apply it to the invoice and send that to the customer to show the reduce in their balance owed to you.
As such I think in CiviCRM we need to fully enforce the split of concepts of the “obligation” and the actual payments (or refunded) amounts. Currently we have started on the journey towards that where Contributions (and line items on them) represent the obligations in most cases, and payments (or financial transactions of type payment) are the payments.
The contribution status should be a reflection of the net of that position. Does the customer owe us money (= pending or partially paid*) or is the balance settled (= completed) or do we owe them money (= pending refund)?
Proposed changes:
- Recalculate the contribution status whenever a contribution is created or updated
We should recalculate the contribution status whenever a contribution is created or updated based on the following rules:
| Ref | Calculation | Status |
|---|---|---|
| 1 | If there are no payments or refunds | Pending (arguably there would be a better label for this but in order to avoid making actual changes to the contribution status options this would seem to be the best fit) |
| 2 | If the sum of the line items (including any tax line items) > (sum of payments received - sum of refunds paid out) | Partially paid |
| 3 | If the sum of the line items (including any tax line items) < (sum of payments received - sum of refunds paid out) | Pending refund |
| 4 | If the sum of the line items (including any tax line items) = (sum of payments received - sum of refunds paid out) | Completed |
- Changes to the User interface:
| Screen | How it looks currently | How it will work | Changes required |
|---|---|---|---|
| New contribution screen |  |
 |
We split the "obligation" with the "payment". As such we remove the contribution status field from this screen and instead allow people to decide whether they want to record the "payment" at the same time as recording the "obligation" by checking the "Record payment" checkbox. The contribution status would then be calculated using the logic above. i. |
| View contribution screen (one line item) |  |
No Changes | |
| View contribution screen (multiple line items) | |||
| Edit contribution screen (one line item) |  |
 |
Status field becomes read only and we introduce a button to cancel the contribution. See below for new screen for cancelling a contribution |
| Edit contribution screen (multiple line items) | |||
| New membership screen | |
cell | ------ |
| View membership screen (one line item) | |||
| View membership screen (multiple line items) | |||
| Edit membership screen (one line item) | |||
| Edit membership screen (multiple line items) | |||
| New event participant screen | |
 |
Change the checkbox "Record Payment" to "Record Contribution" OR have a toggle to allow the admin to decide whether they are recording a "Paid" or "Free ticket". We only need the financial type and contribution date if we are only recording the contribution (note in core#1403 (closed) we have agreed to change label of contribution received date to contribution date). The amounts / lines should come from above. |
| View event participant screen (one line item) | |||
| View event participant screen (multiple line items) | |||
| Edit event participant screen (one line item) | |||
| Edit event participant screen (multiple line items) |
- Remove
Activity
@jamienovick thanks for your proposal. I am in favor of improviving this. But it would probably also be a risky effort as many other extensions work with the contribution status as how they are currently implemented (such as CiviSepa and others).
Hi @jaapjansma thanks for taking a read.
You've jumped the gun a little as it's all still WIP but I've been working with @totten and a few members of the Compuco team on this and one of the guiding principles would be that this would be a UX change only and none of the underlying API's, data structures or option values would change, and as such extensions, reports and integrations can continue to work as they do currently.
It would only be the user experience from an end user perspective in Civi that would be modified, and hopefully to make things far clearer and simpler (think big buttons rather than confusing array of status options).
I hope to have finalised this ticket in the next few weeks then we will be sharing it wider for comment and then organising a call to discuss.
Best
Edited by Jamie Novick - Compuco
added comp:CiviContribute improvement triaged type:proposal labels
I've logged #201 to cover the In Progress & Overdue statuses
Thanks for pushing this along Jamie. I'm in principal in favour of sorting out this problematic field and will try to engage with the details of this proposal.
I echo Jaap's concern about risk. I would suggest a safe general approach is to leave everything as is initially and add new fields to handle new approaches using an extension. Then when the extension has been in production for a while the current core field and its handling can be removed.
As we have discussed several times over the years, this field needs to be refactored because too many things have been overloaded onto it. Originally it was used to track the status of the workflow with payment processor for the single payment that could be related to a contribution. It still has a number of places where that is incorporated but they should all be separated out into workflow statuses for each separate payment and each separate online refund as well as each credit note created.
<Ramble> I would caution that rules vary by jurisdiction on some of these matters. Invoices can be edited in Canada for example (Canada's federal tax authority indicates for example in https://www.canada.ca/en/revenue-agency/services/forms-publications/publications/p-116/collection-gst-a-supplier-where-invoice-silent-on-tax-payable.html the usefulness of this in certain cases where the tax was not initially stated in the invoice). Although consecutive numbering of invoice is good practice, it's not required in some places. </Ramble>
Edited by JoeMurray-
Just an update on this for all - we put in a significant amount of work to implement an extension that:
- disables the status field and calculates it based on the principles above
- adds support for a new entity called credit notes which can be allocated to contributions partially
- decouples the concept of a payment from a contribution fully on the create/edit contribution / record participant and create membership screens.
As such it now means that CiviCRM would have as close to a correct accounting model as possible with the current existing data structures.
See below: https://github.com/compucorp/io.compuco.financeextras
We're just testing it to support 5.75 and would welcome feedback and comments on the extension. Nb. Whilst this is fully tested with the Compuco stack I probably wouldn't suggest putting it into production for yourselves without testing it thoroughly first but you may like to explore it to see what it does.
If over time we can get to a consensus about the way it works, perhaps we can move core in this direction.
