Firstly - sorry for the wall of text. This has ended up being a much longer piece of analysis than expected but I thought I should share it so that we can agree a way ahead.

This all started as we were looking at the best way to implement refunds for a payment processor. Some work was done here to discuss this: #87 but I can see this stalled somewhat - probably due to some of the issues I'm going to discuss here.

Background: The problems with the contribution status field

Just for complete disclosure I hate the CiviCRM contribution status field. Hopefully everyone already knows that. I've probably started discussions on getting rid of it at every CiviCON and documented this on every financial gitlab ticket. It is the biggest blocker we have to reaching an accounting implementation that aligns to standard concepts of accounting.

Why is that?

We appear to have 2 use cases for it, both of which are undocumented.

1. For users to quickly change whether the amounts owed are a) expected to be paid or b) are paid.
2. To reflect the payment status of the contribution i.e. whether the total sum of the line items is >, < or = to the net amount paid in or refunded on that contribution (this is important and I’ll come back to it)

This is a bit of a problem as it’s all a bit chicken and egg! If the user changes the status we need to understand whether we should be creating payments or not (or creating refund payments or not), and if the user records payments or refund payments we need to update the status to something meaningful and consistent.

The following is an analysis of the allowed contribution status changes from and to that a user can perform:

Table 1:

• Black = Option hidden
• X - Val = Option shown but user cannot change to it as we have a validation message
• Y (black) = Option shown, user can select this and I don’t see issues with accounting that occurs
• N/A = Couldn’t test
• Y (red) = Option shown, user can select this and there are issues with accounting that occurs

I’d note that it’s a bit confusing to users to see a value in the dropdown they can’t use. I think we should just hide values they cannot select as this would be much clearer for them.

We end up with some very strange and in some cases incorrect behaviour when allowing users to change the status in a "partially paid" and "pending refund" scenarios:

Table 2:

Ground Rules

As such I would like to suggest some ground rules to work towards:

1. The only time a user can change the status of a contribution should be when it has no payments or refunds attached to it.
2. Once a payment has been received we manage everything through either the line item editor or the “change selections” on the event form or "record payment” or “record refund”. If any changes to each are made to any of these we recalculate the contribution status based on one set of rules:

The rules to calculate the contribution status should be:

Table 3:

Delving deeper, there are a number of situations within CiviCRM that do not confirm to this currently and instead we end up with a status this is confusing for the user: (Note this is not be a comprehensive list - just to give some examples).

Table 4:

Specific Scenarios

This all said there are some specific use cases that we need to consider and have appropriate workflows for:

Table 5:

Actions:

As such I think the actions could be:

1. Agree to the "ground rules" for the contribution status field above so we are all working towards a common goal.
2. I think we should hide values they cannot select as this would be much clearer for them. i.e. all items that are marked as “X-val” in the table above. Perhaps we can refactor the code that show/hides the options to a central place.
3. I think we should change the way the status field is calculated to use the business logic suggested in Table 3 above in all cases. This shouldn’t be something that extensions should take care of but should be calculated any time a contribution haas it’s line items / financial transactions changed. Obviously I don’t know what this means from a code perspective and I assume this would be a significant refactor but I think we could clear up a lot of business logic by doing so.
4. Discuss and agree which of the options from table 5.1 are suitable and then agree the UX for this.
5. So that we can then remove the option to make the status changes as per table 1: 3f, 4e, 4f without degrading users ability to do the things they need.

I actually think if we do these things we can consider CiviCRM’s accounting to be “correct” if not completely intuitive (at least for now).

@mattwire @eileen @KarinG @JoeMurray