Financial issueshttps://lab.civicrm.org/dev/financial/-/issues2022-08-25T21:31:40Zhttps://lab.civicrm.org/dev/financial/-/issues/205CiviFinancial Blue Sky Dreaming2022-08-25T21:31:40ZJoeMurrayCiviFinancial Blue Sky DreamingThere has been talk over the years by different people about a LExIM shift to a new paradigm/implementation for CiviAccount. I have been approached by someone who could rustle up a tiny starting stake for such a huge effort (I'm not conf...There has been talk over the years by different people about a LExIM shift to a new paradigm/implementation for CiviAccount. I have been approached by someone who could rustle up a tiny starting stake for such a huge effort (I'm not confident there would be an ability to get to 50% needed to launch an MIH that will likely cost in the six figures at CT billable rates). Here is a blue sky dreaming of how a CiviFinancials LExIM might succeed.
1. We find funding for integrating payments into Form Builder - this seems likely to be possible and occur, yet is still a big lift. Imagine all the financial aspects of webform_civicrm.
2. When implementing the new interface for contribution pages and event reg pages, etc., we have some 'dependency injection' that allows calls to be made either to the existing financial APIs or to a new set.
3. We work to implement the guts of CiviAccounting in a new way. Discussion is needed here, but I would be in favour of some significant breaking changes like simplifying the Financial Type/Financial Account model into much more standard accounting, preferrably with simple, sensible defaults.
1. We could design our own CiviAccounts v2 from scratch.
2. We could aim to implement an interface with an existing open source accounting system. The aim is to store all the CiviCRM financial transactions in a standard, validated way without having the development and support burden of making it ourselves. We'd only need to keep the integration going strong. Two candidates I have found that might be feasible, and there are many others, are Front Accounting and GnuCash. https://sourceforge.net/projects/frontaccounting/files/FrontAccounting%202.4/stats/timeline?dates=2019-08-01+to+2022-08-01 has the same technical stack as CiviCRM (PHP and MySQL) but seems to be trending down in installs. https://sourceforge.net/projects/gnucash/files/gnucash%20%28stable%29/stats/timeline?dates=2019-08-01+to+2022-08-01 is much more popular and losing momentum more slowly. It is built in C and C++, so it would be more complex to integrate.
3. If there is a fully open source implementation via CiviAccounts v2 or integrating an open source accounting package, then I would be okay with there being integrations with closed source accounting systems that are popular like Xero and QuickBooks Online. I suppose we could allow the existing implementation of CiviAccounts to be the open source option. While more financially realistic, it might be bad for our reputation.https://lab.civicrm.org/dev/financial/-/issues/203False-positive duplicate detection caused by webform not passing correct par...2024-01-16T19:36:50ZAllenShawFalse-positive duplicate detection caused by webform not passing correct parameters to Authorize.net - it needs to pass contributionID(Joinery internal reference: "Trello#142: Membership Application and Payment Process Not Working")
From what I understand so far, it seems there's a well-thought effort to use propertybag to standardize parameter names in payment proces...(Joinery internal reference: "Trello#142: Membership Application and Payment Process Not Working")
From what I understand so far, it seems there's a well-thought effort to use propertybag to standardize parameter names in payment processors. Here's one area where this effort seems incomplete and easily improved:
There are a few places in `CRM_Core_Payment_AuthorizeNet::doPayment()` where directly accessing `$params` could (and does) cause problems; it seems right to switch these to `$propertybag`. (BTW I think that usage of `propertybag` in the context of alterPaymentProcessorParams hook is still not appropriate, so I'm not suggesting changes to that usage within this method.)
**Specific observable problem:**
(I can create a more detailed and stripped-down recipe if needed, but hoping to avoid the effort.)
- On a site with: D7, civicrm 5.49.5, webform_civicrm.module, and contributiontransactlegacy extension
- We have a webform which accepts membership signups, payable through the core Authorize.net payment processor.
- Payments made through this webform are actually transacted in Authorize.net, but the user is given the error, 'It appears that this transaction is a duplicate. Have you already submitted the form once? If so there may have been a connection problem. Check your email for a receipt from Authorize.net. If you do not receive a receipt within 2 hours you can try your transaction again. If you continue to have problems please contact the site administrator.'
- FWIW I have observed that the below corrective measure fixes this problem for this site.
**Why this error happens:**
- `CRM_Core_Payment_AuthorizeNet::doPayment()` calls `$this->checkDupe($authorizeNetFields['x_invoice_num'], $params['contributionID'] ?? NULL)` (see [line 171](https://github.com/civicrm/civicrm-core/blob/a1278f71bd804c8013600793916a692b4345e4a2/CRM/Core/Payment/AuthorizeNet.php#L171)) to detect a duplicate transaction.
- However, `$params['contributionID']` is undefined. (On the other hand, `$params['contribution_id']` is defined, as is `$propertyBag->getContributionID( )`).
- Since `checkDup()` gets a NULL value for `$contributionId`, it always thinks the contribution is a duplicate, so we get the above error message.
**Corrective measueres**
1. Seems like the intent of `propertybag` is to clear up exctly this kind of naming inconsistency. I suggest changing Line 171 to use `$propertyBag->getContributionID( )` instaed of `$params['contributionID']`.
2. [Line 146](https://github.com/civicrm/civicrm-core/blob/a1278f71bd804c8013600793916a692b4345e4a2/CRM/Core/Payment/AuthorizeNet.php#L146), which tries to read `$params['is_recur']` and `$params['contributionRecurID']`, is probably also a good candidate for similar cleanup?
Should I go head with a PR or is more discussion needed?https://lab.civicrm.org/dev/financial/-/issues/150civicrm/payment/form url got empty currency argument in backoffice live CC form2021-07-07T08:47:08ZMonish Debcivicrm/payment/form url got empty currency argument in backoffice live CC formThis affects iATS Payments ACH/EFT payment processor which renders the payment block based on currency chosen. Here are the steps to replicate:
1. Install and enable [iATS paymentextension](https://github.com/iATSPayments/com.iatspayme...This affects iATS Payments ACH/EFT payment processor which renders the payment block based on currency chosen. Here are the steps to replicate:
1. Install and enable [iATS paymentextension](https://github.com/iATSPayments/com.iatspayments.civicrm)
2. Add and configure an iATS Payments ACH/EFT payment type payment processor.
3. Enable USD and CAD currencies
4. Go to 'Submit Credit Card Contribution' backoffice form
5. Choose payment processor created at step 2
Issue:
1. Payment block renders payment block with check template
2. Switching currency doesn't update the payment block.
For more detail discussion on MM - https://chat.civicrm.org/civicrm/pl/ixxrxhxs43r4mfyjpnf6tnwnnc
ping @KarinG @eileen @JoeMurray @seamuslee5.31.0Monish DebMonish Debhttps://lab.civicrm.org/dev/financial/-/issues/130Standardise payment processing2021-05-17T20:44:55ZeileenStandardise payment processingThis is a meta issue for getting more standardisation and clearer best practices in place for payment processing.
Many of our preferred practices are not new but are not clearly & consistently documented and processors that people copy ...This is a meta issue for getting more standardisation and clearer best practices in place for payment processing.
Many of our preferred practices are not new but are not clearly & consistently documented and processors that people copy are not modelling them well.
The key goals at this point are to
1) [model best practices in our core processors](https://lab.civicrm.org/dev/financial/-/issues/132)
2) ensure our docs are up-to-date on best practice
3) build a list of urls of payment processors & create tickets against them to implement the best practices
4) get payment processor writers to subscribe to this meta-issue for updates.
5) start the process of moving the unused core processors to extensions & then entirely out of core.
- done for eway, PayflowPro
6) start to add deprecation notices to functions we do not recommend. Note that these should only show on dev environments as we recommend live sites set appropriate php error level settings. We should make a decent start on 1-4 above in the next 1-2 months and then get more serious about this one
**Best practices to implement in core & promote**
1) All processors should implement doPayment and not doDirectPayment, doTransfer. In order to make this change they need to do 3 things
a) rename the existing function
b) set ```$result['payment_status_id']```
e.g ```$result['payment_status_id'] = CRM_Core_PseudoConstant::getKey('CRM_Contribute_BAO_Contribution', 'contribution_status_id', 'Completed');```
c) handle $params['amount'] == 0 (generally return at that point, setting payment_status_id)
d) throw exceptions on failure - do NOT return CRM_Core_Error
2) Handle the settings bag
Processors should make their code able to handle the settings bag. The settings bag works like an array except that
1) - some array functions don't work - eg array_merge()
2) - it standardises input.
In order to prepare processors should
1) Add the line ``` $propertyBag = PropertyBag::cast($params);``` as the first line it doPayment to ensure that they are consistently working with a propertyBag object
2) replace array functions with functions that work on the property bag - eg replace empty() checks with $params->has('recurProcessorID')
Note that we will add to this as we convert core processors & identify the difficulties.
3) Not extend BaseIPN & work to eliminate that class. currently we recommend processors implement ```handlePaymentNotification()``` rather than have an IPN class per se but the core processors don't do that. We should model 'good behaviour' in our core processors & document. Currently our core processors are a bit nasty - subtasks on this are
- Getting a clear api/ expectations for cancel & failed payments out of BaseIPN
3) Use guzzle not CURL & use GuzzleTestTrait to add tests. Note that we should focus on moving weird ones like FirstData outside of core but it's proven very easy to switch A.net over & we want to model this https://lab.civicrm.org/dev/financial/-/issues/143https://lab.civicrm.org/dev/financial/-/issues/95Update Payment.create api handling of payment instrument2019-11-01T12:04:03ZeileenUpdate Payment.create api handling of payment instrumentAt the moment the passed in payment_instrument_id will take precedent over the one associated with the relevant processor. I don't think that is how the design was intended and in other places we use the paymnet_processor_id.
Note that...At the moment the passed in payment_instrument_id will take precedent over the one associated with the relevant processor. I don't think that is how the design was intended and in other places we use the paymnet_processor_id.
Note that I think that the Payment.create api should require either the payment_instrument_id or the payment_processor_id.
Logically this is something 'known' when making a payment & not something we should 'guess'. Transitionally we would deprecate NOT passing in one of those values rathe than requiring it.
I do think we should add some handling to Order.create so that specifically when chaining Payment.create it automatically sets some values - this being one.
@JoeMurray @monish.deb @artfulrobothttps://lab.civicrm.org/dev/financial/-/issues/93Agree / implement handling of failure for doPayment/ doRefund / params for do...2019-11-01T11:58:47ZeileenAgree / implement handling of failure for doPayment/ doRefund / params for doRefundCurrently both doPayment and doRefund throw a PaymentProcessorException if there is a failure either in terms of config or in terms of processing. The flow we agreed back when we implemented this in 4.6 is we should be working towards
`...Currently both doPayment and doRefund throw a PaymentProcessorException if there is a failure either in terms of config or in terms of processing. The flow we agreed back when we implemented this in 4.6 is we should be working towards
```
Order.create.... status=pending
try {
$result = $paymentObject->doPayment();
if ($result['payment_status_id') === 1) {
Payment.create.....
}
}
else {
\\ Add code here to add failed payment
}
```
Currently the CRM_Core_Payment doPayment converts failed results for old processors to exceptions and throws them and new processors have been expected to throw exceptions.
Some of the forms implement things similar to the above although in other cases the exception is caught much further from the call to doPayment and in some cases we are still not creating the order before processing the payment.
Note that in this flow a payment that did not fail but was not successful would not result in a payment being created.
@mattwire is proposing that we change this for doRefund and stop throwing exceptions for failed payments (or any types) and instead return a status for failed for user-related reasons and an exception for system related reasons.
The options as I see them are
1) implement the same pattern for doRefund as doPayment
2) implement and extend the pattern - ie add a couple of things we probably would have done if implementing from scratch such as
- a PaymentProcessorException_PaymentFailed Exception which extends the existing PaymentProcessorException & can be throwing when the payment relates to card failure rather than a config error. Calling code can choose to catch these separately if it chooses. (we could use this for doPayment too)
- add a getter for the outcome ie. start a transition from returning 1 / the value that maps to contributionStatus = Completed (which is not actually a payment status) to ```$paymentObject->isCompleted()``` or ``` $paymentObject->isSuccessful()```. (Note Omnipay uses isSuccessful()). We could transition doPayment to this over time.
- add a property / getter for getRefundTrxnResult
- add a property / getter for 'processor_result' / 'passthrough_params' which is something that @mattwire has suggested although I have no specifics / examples.
- encourage the use of getters & setters rather than passing in params - ie ```$paymentObject->setAmount(20.40);``` Notte we have just added getters & setters to the class in general https://lab.civicrm.org/dev/financial/issues/82 - implement this in the api call & new core interactions with the class
3) switch to the pattern matt proposes - ie. for doRefund
```
return [
// refund_status_id would normally return the contribution_status_id Completed or Failed
'refund_status_id' => CRM_Core_PseudoConstant::getKey('CRM_Contribute_BAO_Contribution', 'contribution_status_id', 'Completed'),
// The refund trxn_id may be different from the trxn_id of the original payment.
// If it is the same then trxn_id should be copied to refund_trxn_id
'refund_trxn_id' => NULL,
// Array of params returned by the payment processor. The contents of this will vary by processor and should not be relied on.
// However, they can be very useful for logging or providing specific feedback.
'processor_result' => [],
];
```https://lab.civicrm.org/dev/financial/-/issues/90Add new api Order functionality to update line items2019-10-25T08:12:07ZeileenAdd new api Order functionality to update line itemsI'm pretty sure that despite @JoeMurray's optimism the current apiv3 Order.create doesn't support this. @monish.deb & I discussed adding it back here
https://github.com/civicrm/civicrm-core/pull/11380
I don't think we should add it o a...I'm pretty sure that despite @JoeMurray's optimism the current apiv3 Order.create doesn't support this. @monish.deb & I discussed adding it back here
https://github.com/civicrm/civicrm-core/pull/11380
I don't think we should add it o apiv3 now - I think we should write a v4 api that does it. I think the object oriented approach will allow us to come up with cleaner methods. I'm kinda keen to try to build those up on a ORDER pseudoBAO class & play a bit before we expose in the api so we can learn about about what works & doesn't before locking it inhttps://lab.civicrm.org/dev/financial/-/issues/87Partial Refunds2022-06-14T18:38:44Zmattwiremjw@mjwconsult.co.ukPartial RefundsStripe supports partial refunds via the Stripe Dashboard. I think other processors support similar.
To record a partial refund in CiviCRM we can record a negative payment on the contribution using API `Payment.create`.
Currently nothin...Stripe supports partial refunds via the Stripe Dashboard. I think other processors support similar.
To record a partial refund in CiviCRM we can record a negative payment on the contribution using API `Payment.create`.
Currently nothing changes on the contribution:
* Total amount still shows the full amount but payments are recorded correctly so the sum of payments does not equal the contribution total amount.
* Contribution status remains Completed.
What should happen(?):
* A partial refund means that the total_amount paid, tax_amount and the fee_amount paid may be reduced.
* The contribution status is no longer "Completed" and should be `Partially refunded` (does not exist)? `Partially paid` (already exists).
* The `Financial Type` should be displayed correctly for the refund payment.
Currently, viewing a contribution, either in the summary or detail gives no indication of the actual amount paid:
![image](/uploads/39a50dd48d946207feda438321850a39/image.png)
![image](/uploads/63cc3abf7df50ac1babcbff85f2d4c86/image.png)
@JoeMurray @eileen @artfulrobot @ayduns Thoughts please?https://lab.civicrm.org/dev/financial/-/issues/74Order.create - registering multiple Participants (participantPayment records ...2020-03-04T23:10:14ZAndrei Mondocandreimondoc@gmail.comOrder.create - registering multiple Participants (participantPayment records not created properly)Creating an Order to register two (or more) participants creates two (or more) `ParticipantPayment` records, one for each participant line item, registering two participants via an Event page creates only one `ParticipantPayment` record ...Creating an Order to register two (or more) participants creates two (or more) `ParticipantPayment` records, one for each participant line item, registering two participants via an Event page creates only one `ParticipantPayment` record regardless of the number of participants registered.
Which is the correct one/what is expected behaviour?
### Steps to reproduce
Call Order.create with parameters similar to:
``` php
$params = [
'total_amount' => 100.00,
'financial_type_id' => 4,
'contribution_status_id' => 'Pending',
'payment_instrument_id' => 1,
'currency' => 'USD',
'source' => 'Multiple participants (Order.create)',
'contact_id' => 2,
'line_items' => [
1 => [
'line_item' => [
0 => [
'price_field_id' => 3,
'label' => 'Tickets',
'financial_type_id' => 4,
'price_field_value_id' => 3,
'qty' => 1,
'field_title' => '1 Ticket',
'unit_price' => 50.000000000,
'line_total' => 50,
'entity_table' => 'civicrm_participant',
],
],
'params' => [
'contact_id' => 2,
'event_id' => 1,
'role_id' => 1,
'source' => 'Multiple participants (Order.create)',
'price_set_id' => 7,
'fee_level' => '1 Ticket',
'fee_amount' => 50,
],
],
2 => [
'line_item' => [
0 => [
'price_field_id' => 3,
'label' => 'Tickets',
'financial_type_id' => 4,
'price_field_value_id' => 3,
'qty' => 1,
'field_title' => '1 Ticket',
'unit_price' => 50.000000000,
'line_total' => 50,
'entity_table' => 'civicrm_participant',
],
],
'params' => [
'contact_id' => 5,
'event_id' => 1,
'role_id' => 1,
'source' => 'Multiple participants (Order.create)',
'price_set_id' => 7,
'fee_level' => '1 Ticket',
'fee_amount' => 50,
],
],
],
];
$order = civicrm_api3('Order', 'create', $params);
```
This results in two ParticipantPayment records:
``` json
// civicrm_participant_payment
[
{
"id": 5,
"participant_id": 5,
"contribution_id": 18
},
{
"id": 6,
"participant_id": 6,
"contribution_id": 18
}
]
```
Registering two or more participants via an Event page, always creates one ParticipantPayment regardless of the number of participants registered.
``` json
// civicrm_participant_payment
{
"id": 7,
"participant_id": 7,
"contribution_id": 19
}
```