​

Context

We are using Stripe Subscriptions, meaning that Stripe is responsible for creating the subscriptions and managing the retries of the failing methods. At the beginning we want to mimic the Stripe retry logic to have similar recovery results.

​

Solution

​

Approach 1: New Dunning concept

This approach involved creating a new DunningAttempt model that keeps track of the failed payment attempts for subscriptions. A dedicated worker would manage the state transitions. Overview:

1. New DunningAttempt model: Introduce a new class and table to process subscription failed payments. This new class will have:
   1. id: UUID
   2. order_id: foreign key to Order (unique - we should have only 1 row per order)
   3. status: retrying, succeeded, failed
   4. attempt_number: the current retry count (from 1 through 4)
   5. next_payment_attempt_at: date when the next retry is scheduled
   6. last_failure_reason: the error message that failed
   7. started_at: when the dunning process began
2. Renewal Job: A scheduled job (subscription.cycle) runs regularly to find subscriptions due for renewal.
3. Payment Attempt: For each due order subscription, it calls PaymentService to attempt a charge.
4. State Transition:
   • On Failure: The subscription status is changed to past_due, and a new DunningAttempt record is created with attempt_number set to 1, status retrying, and a next_payment_attempt_at. The benefits will be revoked.
   • On Success: The subscription is renewed, and the status remains active.
5. Dunning Worker: A new periodic worker (subscription.dunning) runs hourly.
   • It queries for dunning_attempts with status = "retrying", and next_payment_attempt_at it’s in the past.
   • It re-attempts payment. If it fails again, it updates next_payment_attempt_at for the next retry.
   • After a configured number of retries, it moves the DunningAttempt to failed and the subscription to unpaid or canceled.
6. Recovery: If a payment succeeds during the dunning process, the subscription status is set back to active and the DunningAttempt is marked to succeeded

​

Sequence Diagram

First payment Dunning retries TBD: User updated payment method TODO

​

Approach 2: Store the retries in the Subscription model

I discarded this solution as it increases the complexity of the Subscription model.

​

(Recommended) Approach 3: Using Order and Payment Models

This approach relies on the existing Order and Payment models, making it a lightweight and integrated solution. It avoids a new table only needing minimal changes to the Order model. Overview:

1. Change Order Model: Add a single, nullable timestamp field to the Order model:
   • next_payment_attempt_at: Schedules the next retry. If NULL, no retry is pending.
2. Use Existing Payment Model: The Payment model, which already has a foreign key to Order, will serve as the log for all payment attempts.
   • attempt_number can be derived by counting the Payment records associated with the order.
   • last_failure_reason can be retrieved from the most recent failed Payment record.
3. Renewal and Initial Payment: The process starts as before. A scheduled job creates an Order and a Payment is attempted.
4. State Transition on Failure:
   • When a payment fails, the Payment record is marked as failed.
   • The Order is updated by setting next_payment_attempt_at to schedule the first retry (e.g., 3 days from now).
   • The Subscription status is set to past_due. Benefits are revoked.
5. Dunning Worker: A periodic worker (order.process_dunning) runs hourly.
   • It queries for Orders where next_payment_attempt_at is not NULL and is in the past.
   • For each due order, it creates a new Payment record and attempts a charge.
   • On Failure: It reschedules next_payment_attempt_at for the next attempt.
   • On Final Failure: After the last attempt, next_payment_attempt_at is set to NULL. The Subscription is moved to unpaid or canceled.
6. Recovery:
   • If a payment succeeds, the Payment is marked as succeeded, the Order is marked as paid, and next_payment_attempt_at is set to NULL.
   • The Subscription status is set back to active. Benefits are re-granted.

​

Comparison with Approach 1

Pros:

• Maintainability: Avoids a new table and only adds one nullable column to Order.
• Leverages Existing Models: Reuses the Payment model for its intended purpose, keeping the design clean and logical.
• Clear and Explicit: The Order model is only concerned with when the next payment is due, while the Payment model correctly stores the history of what happened.

Cons:

• Complex Queries: Retrieving the full dunning history for an order (like the attempt count or last error) requires querying the associated Payment records. We only expect 4 payments per order. Mitigation: it can be cached or denormalized if it becomes expensive.
• Race conditions: As before, we can have a Race condition when the retry mechanism is working and the user plans to manually pay. Mitigation can be prevented by locking the Order.

​

Sequence Diagrams

First Subscription Payment Failure This diagram shows the flow when a recurring subscription payment fails for the first time, initiating the dunning process. Dunning Retries This diagram illustrates how the periodic worker handles scheduled payment retries. User Updates Payment Method TODO

​

Questions

How does the retry mechanism look in our Stripe account In Stripe we have:

• 4 retries at most.
• These 4 retries should happen before 3 weeks.
• If all retries failed, we set the status of the subscription to canceled.

How many payments got recovered? Results:

• 2682 with at least 1 failed payment
• 1108 recovered orders
• Then 41% of recovery rate.

Results without Premiumize

• 1253 total
• 541 recovered
• Then 43% of recovery rate.

What should we do with the benefits when the first payment of a subscription fails? The current behavior in our platform is to set the subscription status as past due and revoke the benefits. We will follow this approach as of now. Should we expose the retries to the API and to the user? Stripe shows the payment attempts under the invoice and payment_intent objects. Each payment intent can have multiple transactions associated with it. From what I see, we don’t expose these payment intents in our API. I will keep this as a separate task if we want to do it. What happens when the customer updates the payment method? The customer can trigger a manual payment action. If the payment succeeds the order will be marked as paid; if it fails, it will follow the original dunning schedule. What is the backoff strategy of Stripe? Stripe has different backoffs, but based on (t1, t2, t3) we have the following ranges:

1. (D+0) Subscription payment fails
2. (D+2/3) First attempt
3. (D+7/12) Second attempt
4. (D+14/17) Third attempt
5. (D+21) Fourth attempt.

I will go with the earliest ranges. If the order goes unpaid, should we move the subscription status to unpaid or canceled? In Stripe, we are moving it to canceled.