Proposal - store metadata on membership renewal on line item

UPDATE - discussion on this is converging on adding a new json field 'metdata' to the line item table, permitting data related to that line item to be stored at point of order, and used when confirming. (UPDATE - this is not really the convergence now - proposal updated below)

We have a few issues open ( #28 (moved) , core#1402 (closed) , financial#53 ) that are to my mind blocked by us not having a data model that holds the user's intent when renewing. The focus of this gl is the data model and exposing it via the back office renewal form & doing what is needed in the apis.

The problem

When renewing an expired membership the back office user has a fairly blunt way of influencing the calculated dates - they can set the 'renewal date'. However, if the payment is not made in the same form submission that intent is not captured anywhere and cannot be respected when completing the transaction. Likewise the receipt text and number of renewal terms are not captured.

The proposal data model

1. add membership_num_terms to civicrm_line_item table.
2. add 'payment_completion_metadata' to the civicrm_line_item civicrm_contribution table for data that is only used in the order->payment flow. Limit what can be stored here to tested values used in that flow - likely to be the message details and effective date the renewal form uses but doesn't sotre.

1. New field added to civicrm_line_item called 'metadata' which stores information required to complete the membership renewal - e.g. the field might store ``json_encode(['start_date' => '2010-01-01', 'end_date' => '2020-01-01', 'membership_status_id' => 'Current', 'membership_num_terms' => 2']

- note this is the new proposal - but I'm wondering what happens if it's renewed in the meantime & then paid

- New status 'Pending renewal' used when there is a need to record the dates (is_current_member = 0, is_reserved = 1, is_admin = 1 too I think) - When a contribution linked to a membership in 'Pending renewal' is completed the status is updated but the dates are not changed - If a membership has 'Pending renewal' status and status_override_end_date is set then any processing of that membership after that date would result in it being reverted to previous dates and status - this information should be available in the membership_log table.

Proposed form exposure

No proposed form changes - this is just storing data already submitted in the renewal form.

~~- the logic could be used outside of existing core forms but within core I think this is something we are exposing to the back-office user doing a renewal - they already have considerable ability to edit dates. So I think this is a form layer thing not some site-wide setting.

• when renewing an expired membership the renewal form would present the user with what will happen to the start date, end date etc if payment is received on that day and offer a checkbox to specify dates - if that is checked then date fields would be exposed an on save the Pending Renewal status would be used with the selected dates.
• potentially the user can also enter the date the 'offer' is available until - ie status_override_end_date~~

Edge cases

1. Multiple terms

We would add an extra field membership_num_terms to the line items column #28 (moved) throws up the edge case that more than one renewal term might be selected. Both @jptillman and I assumed that the right way to represent that would be setting qty = n and then picking that when confirming. @andrewhunt didn't think that was the right assumption https://github.com/civicrm/civicrm-core/pull/18618. If we come down on NOT setting qty then we ALSO need a way to record intent for non-expired renewals - this could be an extra membership_num_terms column on the line_item table. (I'm still inclined to my original assumption but more points of view needed).

2. About to expire rather than actually expired

Out of scope for now - just do what the form currently does

I think we would still manage this in the form ie - "the membership is due to to expire in 4 days, if payment is received after that then .... ' and present the same options as for an actually expired membership