Transaction IDs for a contribution that are initially null are always null

This ticket is the result of the inquiry in this other core#468 (closed)

"Normal behaviour"

1. Create a contribution with transaction ID

2. Visualize contribution, transaction ID is present

3. Edit payment details and set a different transaction ID

4. Visualize contribution, new transaction ID is appended to the old transaction ID

Buggy behaviour

1. Create a contribution with transaction ID null

2. Edit payment details and set a transaction ID value

3. Visualize contribution, transaction ID isn't present

mentioned in issue core#1011 (closed)

So ideally we would hide that & deprecate that field from the contribution make the payment trxn ids more visible

Note then, that searches and reports are looking the transaction id at contribution level, not at payment level. On the example of the description, if you search the transaction ID 222222 you do not get any results.

We also need to be clear what is retrieved when searching for a specific transaction ID using the contribution record (eg. via API) - it sounds like it would make most sense to retrieve the latest transaction ID recorded as a payment.

It would make more sense for me search transaction id on any payment. That is, if I want to identify a payment through an identifier, I want to search even if it is not the last one associated with the contribution. For example, when I look for an original payment in a contribution that has occurred a refund. Or also in the case of partial payments.

@francescbassas Yes makes sense.

@mattwire I've been thinking the way to tackle this is to add an (initially hidden) setting to opt into suppressing contribution level payment fields & exposing them on payments. Once it's mature we can grandfather it in - starting with new installs.

added needs:concept approval sig:bug triaged labels

@eileen It's still not completely clear to me what fields are supposed to be where - it's all a bit of a mess! In conjunction with #53 and #57 we need to identify what we actually need it to do and what we'd like it to do before tweaking stuff in core.

@mattwire the contribution.trxn_id field is basically deprecated IMHO - IATS puts stuff there but that's mostly to compensate for the lack of visibility on payment.trxn_id. It was because of IATS & out of safety concern I though grandfathering in makes more sense. Note this applies to

trxn_id, payment_instrument_id, check_number

I think this is too complex to just deprecate them without thinking about how we actually need it to work.

On a contribution we have trxn_id and invoice_id. Both are globally unique - if they are used to store a processor reference there is always a risk of duplicates (not with IATS because the references are CiviCRM side, but with (let's say 50%) of processors the reference is processor side). So we actually need to be unique per payment processor + reference. In the case of Stripe we have a contribution level reference (the invoice ID) and one or more payment level references (the "charge" IDs). Then we also need to make sure we can actually save the reference on the contribution without doing crazy/wierd stuff (currently the trxn_id on the contribution is saved most of the time but sometimes it's not). And I'm pretty sure the invoice_id field is auto-generated and impossible to save in quite a few of the payment flows.

And I need to be able to easily identify a contribution and/or payment when I'm given a reference by the processor (eg. a Stripe IPN - they're not just used for recurring).

By the way, I still pretty much work at the contribution level, using Contribution.create, Contribution.completetransaction/repeattransaction or passing back values from doPayment() and hoping CiviCRM does something nice with them. I don't really know how/where Payments fit into that and before we make any changes around existing fields we need to be really clear where we need to go.

And also bear in mind that many sites have multiple different processors in use (eg. a site running IATS and Stripe) that may have different requirements (ie. one has a CiviCRM generated unique ID, the other an external "unique" ID).

Then on recurring we have processor_id, trxn_id, payment_token_id, invoice_id. But in reality you have to use processor_id because that is (or at least was) the only value passed to one of the updateSubscription/cancelSubscription functions - except it gets renamed to subscriptionId before it gets there... Confused yet?

I think this is too complex to just deprecate them without thinking about how we actually need it to work.

On a contribution we have trxn_id and invoice_id. Both are globally unique - if they are used to store a processor reference there is always a risk of duplicates (not with IATS because the references are CiviCRM side, but with (let's say 50%) of processors the reference is processor side). So we actually need to be unique per payment processor + reference. In the case of Stripe we have a contribution level reference (the invoice ID) and one or more payment level references (the "charge" IDs). Then we also need to make sure we can actually save the reference on the contribution without doing crazy/wierd stuff (currently the trxn_id on the contribution is saved most of the time but sometimes it's not). And I'm pretty sure the invoice_id field is auto-generated and impossible to save in quite a few of the payment flows.

And I need to be able to easily identify a contribution and/or payment when I'm given a reference by the processor (eg. a Stripe IPN - they're not just used for recurring).

By the way, I still pretty much work at the contribution level, using Contribution.create, Contribution.completetransaction/repeattransaction or passing back values from doPayment() and hoping CiviCRM does something nice with them. I don't really know how/where Payments fit into that and before we make any changes around existing fields we need to be really clear where we need to go.

And also bear in mind that many sites have multiple different processors in use (eg. a site running IATS and Stripe) that may have different requirements (ie. one has a CiviCRM generated unique ID, the other an external "unique" ID).

Then on recurring we have processor_id, trxn_id, payment_token_id, invoice_id. But in reality you have to use processor_id because that is (or at least was) the only value passed to one of the updateSubscription/cancelSubscription functions - except it gets renamed to subscriptionId before it gets there... Confused yet?

@mattwire right deprecate means obscure in the UI & show payment fields instead - I don't think we can aspire to remove the field at any point

I leave as reference Eileen's blog post https://civicrm.org/blog/eileen/searching-payments-civi which is connected in some way with this discussion.