Idempotent payment webhooks checklist for safe billing updates

Why payment webhooks create duplicate updates

A payment webhook is a message your payment provider sends to your backend when something important happens, like a charge succeeds, an invoice is paid, a subscription renews, or a refund is issued. It’s basically the provider saying, “Here’s what happened. Update your records.”

Duplicates happen because webhook delivery is designed to be reliable, not exactly once. If your server is slow, times out, returns an error, or is briefly unavailable, the provider will usually retry the same event. You can also see two different events that refer to the same real-world action (for example, an invoice event and a payment event tied to one payment). Events can arrive out of order too, especially with fast follow-ups like refunds.

If your handler isn’t idempotent, it can apply the same event twice, which turns into problems customers and finance teams notice right away:

• An invoice marked paid twice, creating duplicate accounting entries
• A renewal applied twice, extending access too far
• Entitlements granted twice (extra credits, seats, or features)
• Refunds or chargebacks not reversing access correctly

This isn’t just “best practice.” It’s the difference between billing that feels dependable and billing that creates support tickets.

The goal of this checklist is simple: treat each incoming event as “apply at most once.” You’ll store a stable identifier for every event, handle retries safely, and update invoices, subscriptions, and entitlements in a controlled way. If you’re building the backend in a no-code tool like AppMaster, the same rules still apply: you need a clear data model and a repeatable handler flow that stays correct under retries.

Idempotency basics you can apply to webhooks

Idempotency means processing the same input more than once produces the same final state. In billing terms: one invoice ends up paid once, one subscription updates once, and access is granted once, even if the webhook is delivered twice.

Providers retry when your endpoint times out, returns a 5xx, or the network drops. Those retries repeat the same event. That’s different from a separate event that represents a real change, like a refund days later. New events have different IDs.

To make this work, you need two things: stable identifiers and a small “memory” of what you’ve already seen.

What IDs matter (and what to store)

Most payment platforms include an event ID that’s unique to the webhook event. Some also include a request ID, idempotency key, or a unique payment object ID (like a charge or payment intent) inside the payload.

Store what helps you answer one question: “Have I already applied this exact event?”

A practical minimum:

• Event ID (unique key)
• Event type (useful for debugging)
• Received timestamp
• Processing status (processed/failed)
• Reference to the affected customer, invoice, or subscription

The key move is to store the event ID in a table with a unique constraint. Then your handler can do this safely: insert event ID first; if it already exists, stop and return 200.

How long to keep dedupe records

Keep dedupe records long enough to cover late retries and support investigations. A common window is 30 to 90 days. If you deal with chargebacks, disputes, or longer subscription cycles, keep them longer (6 to 12 months), and purge old rows so the table stays fast.

In a generated backend like AppMaster, this maps cleanly to a simple WebhookEvents model with a unique field on the event ID, plus a business process that exits early when a duplicate is detected.

Design a simple data model for deduping events

A good webhook handler is mostly a data problem. If you can record each provider event exactly once, everything that follows gets safer.

Start with one table that acts like a receipt log. In PostgreSQL (including when modeled in AppMaster’s Data Designer), keep it small and strict so duplicates fail fast.

The minimum you need

Here’s a practical baseline for a webhook_events table:

• provider (text, like "stripe")
• provider_event_id (text, required)
• status (text, like "received", "processed", "failed")
• processed_at (timestamp, nullable)
• raw_payload (jsonb or text)

Add a unique constraint on (provider, provider_event_id). That single rule is your main dedupe guardrail.

You’ll also want the business IDs you use to locate records to update. These are different from the webhook event ID.

Common examples include customer_id, invoice_id, and subscription_id. Keep them as text because providers often use non-numeric IDs.

Raw payload vs parsed fields

Store the raw payload so you can debug and reprocess later. Parsed fields make queries and reporting easier, but only store what you actually use.

A simple approach:

• Always store raw_payload
• Also store a few parsed IDs you query often (customer, invoice, subscription)
• Store a normalized event_type (text) for filtering

If an invoice.paid event arrives twice, your unique constraint blocks the second insert. You still have the raw payload for audits, and the parsed invoice ID makes it easy to locate the invoice record you updated the first time.

Step by step: a safe webhook handler flow

A safe handler is boring on purpose. It behaves the same way every time, even when the provider retries the same event or delivers events out of order.

The 5-step flow to follow every time

1. Verify the signature and parse the payload. Reject requests that fail signature checks, have an unexpected event type, or can’t be parsed.

2. Write the event record before touching billing data. Save the provider event ID, type, created time, and the raw payload (or a hash). If the event ID already exists, treat it as a duplicate and stop.

3. Map the event to a single “owner” record. Decide what you’re updating: invoice, subscription, or customer. Store external IDs on your records so you can look them up directly.

4. Apply a safe state change. Only move state forward. Don’t undo a paid invoice because a late “invoice.updated” arrives. Record what you applied (old state, new state, timestamp, event ID) for audit.

5. Respond quickly and log the outcome. Return success once the event is safely stored and either processed or ignored. Log whether it was processed, deduped, or rejected, and why.

In AppMaster, this usually becomes a database table for webhook events plus a Business Process that checks “seen event ID?” and then runs the minimal update steps.

Handling retries, timeouts, and out of order delivery

Providers retry webhooks when they don’t get a fast success response. They may also send events out of order. Your handler needs to stay safe when the same update arrives twice, or a later update arrives first.

One practical rule: reply fast, do the work later. Treat the webhook request as a receipt, not a place to run heavy logic. If you call third-party APIs, generate PDFs, or recalculate accounts inside the request, you increase timeouts and trigger more retries.

Out of order: keep the newest truth

Out-of-order delivery is normal. Before applying any change, use two checks:

• Compare timestamps: only apply an event if it’s newer than what you’ve already stored for that object (invoice, subscription, entitlement).
• Use status priority when timestamps are close or unclear: paid beats open, canceled beats active, refunded beats paid.

If you already recorded an invoice as paid and a late “open” event arrives, ignore it. If you received “canceled” and later an older “active” update appears, keep canceled.

Ignore vs queue

Ignore an event when you can prove it’s stale or already applied (same event ID, older timestamp, lower status priority). Queue an event when it depends on data you don’t have yet, like a subscription update arriving before the customer record exists.

A practical pattern:

• Store the event immediately with a processing state (received, processing, done, failed)
• If dependencies are missing, mark it as waiting and retry in the background
• Set a retry limit and alert after repeated failures

In AppMaster, this is a good fit for a webhook events table plus a Business Process that acknowledges the request quickly and processes queued events asynchronously.

Safely updating invoices, subscriptions, and entitlements

Once you’ve handled deduplication, the next risk is split billing state: the invoice says paid, but the subscription is still past due, or access was granted twice and never revoked. Treat every webhook as a state transition and apply it in one atomic update.

Invoices: make status changes monotonic

Invoices can move through states like paid, voided, and refunded. You may also see partial payments. Don’t “toggle” an invoice based on whatever event arrived last. Store the current status plus key totals (amount_paid, amount_refunded) and only allow forward-safe transitions.

Practical rules:

• Mark an invoice paid only once, the first time you see a paid event.
• For refunds, increase amount_refunded up to the invoice total; never decrease it.
• If an invoice is voided, stop fulfillment actions, but keep the record for audit.
• For partial payments, update amounts without granting “fully paid” benefits.

Subscriptions and entitlements: grant once, revoke once

Subscriptions include renewals, cancellations, and grace periods. Keep subscription status and period boundaries (current_period_start/end), then derive entitlement windows from that data. Entitlements should be explicit records, not a single boolean.

For access control:

• One entitlement grant per user per product per period
• One revocation record when access ends (cancellation, refund, chargeback)
• An audit trail that records which webhook event caused each change

Use one transaction to avoid split states

Apply invoice, subscription, and entitlement updates in a single database transaction. Read current rows, check whether this event was already applied, then write all changes together. If anything fails, roll back so you don’t end up with “paid invoice” but “no access,” or the reverse.

In AppMaster, this often maps well to a single Business Process flow that updates PostgreSQL in one controlled path and writes an audit entry alongside the business change.

Security and data safety checks for webhook endpoints

Webhook security is part of correctness. If an attacker can hit your endpoint, they can try to create fake “paid” states. Even with deduplication, you still need to prove the event is real and keep customer data safe.

Verify the sender before you touch billing data

Validate the signature on every request. For Stripe, that typically means checking the Stripe-Signature header, using the raw request body (not a rewritten JSON), and rejecting events with an old timestamp. Treat missing headers as a hard failure.

Validate basics early: correct HTTP method, Content-Type, and required fields (event id, type, and the object id you’ll use to locate an invoice or subscription). If you build this in AppMaster, keep the signing secret in environment variables or secure config, never in the database or client code.

A quick security checklist:

• Reject requests without a valid signature and fresh timestamp
• Require expected headers and content type
• Use least-privilege database access for the webhook handler
• Store secrets outside your tables (env/config), rotate when needed
• Return 2xx only after you persist the event safely

Keep logs useful without leaking secrets

Log enough to debug retries and disputes, but avoid sensitive values. Store a safe subset of PII: provider customer ID, internal user ID, and maybe a masked email (like a***@domain.com). Never store full card data, full addresses, or raw authorization headers.

Log what helps you reconstruct what happened:

• Provider event id, type, created time
• Verification result (signature ok/failed) without storing the signature
• Dedupe decision (new vs already processed)
• Internal record IDs touched (invoice/subscription/entitlement)
• Error reason and retry count (if you queue retries)

Add basic abuse protection: rate limit by IP and (when possible) by customer ID, and consider allowing only known provider IP ranges if your setup supports it.

Common mistakes that cause double charges or double access

Most billing bugs aren’t about math. They happen when you treat a webhook delivery like a single, reliable message.

Mistakes that most often lead to duplicated updates:

• Deduping by timestamp or amount instead of event ID. Different events can share the same amount, and retries can arrive minutes later. Use the provider’s unique event ID.
• Updating your database before verifying the signature. Verify first, then parse, then act.
• Treating every event as the source of truth without checking current state. Don’t blindly mark an invoice paid if it’s already paid, refunded, or void.
• Creating multiple entitlements for the same purchase. Retries can create duplicate rows. Prefer an upsert like “ensure entitlement exists for subscription_id,” then update dates/limits.
• Failing the webhook because a notification service is down. Email, SMS, Slack, or Telegram shouldn’t block billing. Queue notifications and still return success after core billing changes are safely stored.

A simple example: a renewal event arrives twice. The first delivery creates an entitlement row. The retry creates a second row, and your app sees “two active entitlements” and grants extra seats or credits.

In AppMaster, the fix is mostly about flow: verify first, insert the event record with a unique constraint, apply billing updates with state checks, and push side effects (emails, receipts) to async steps so they can’t trigger a retry storm.

Realistic example: duplicate renewal + later refund

This pattern looks scary, but it’s manageable if your handler is safe to rerun.

A customer is on a monthly plan. Stripe sends a renewal event (for example, invoice.paid). Your server receives it, updates the database, but takes too long to return a 200 response (cold start, busy database). Stripe assumes it failed and retries the same event.

On the first delivery, you grant access. On the retry, you detect it’s the same event and do nothing. Later, a refund event arrives (for example, charge.refunded) and you revoke access once.

Here’s a simple way to model state in your database (tables you can build in AppMaster Data Designer):

• webhook_events(event_id UNIQUE, type, processed_at, status)
• invoices(invoice_id UNIQUE, subscription_id, status, paid_at, refunded_at)
• entitlements(customer_id, product, active, valid_until, source_invoice_id)

What the database should look like after each event

After Event A (renewal, first delivery): webhook_events gets a new row for event_id=evt_123 with status=processed. invoices is marked paid. entitlements.active=true and valid_until moves forward one billing period.

After Event A again (renewal, retry): insertion into webhook_events fails (unique event_id) or your handler sees it already processed. No changes to invoices or entitlements.

After Event B (refund): a new webhook_events row for event_id=evt_456. invoices.refunded_at is set and status=refunded. entitlements.active=false (or valid_until is set to now) using source_invoice_id to revoke the right access once.

The important detail is timing: the dedupe check happens before any grant or revoke writes.

Pre launch quick checklist

Before you turn on live webhooks, you want proof that one real-world event updates billing records exactly once, even if the provider sends it twice (or ten times).

Use this checklist to validate your setup end to end:

• Confirm every incoming event is saved first (raw payload, event id, type, created time, and signature verification result), even if later steps fail.
• Verify duplicates are detected early (same provider event id) and the handler exits without changing invoices, subscriptions, or entitlements.
• Prove the business update is one-time only: one invoice status change, one subscription state change, one entitlement grant or revoke.
• Make sure failures are recorded with enough detail to replay safely (error message, step that failed, retry status).
• Test that your handler returns a response fast: acknowledge receipt once stored, and avoid slow work inside the request.

You don’t need a big observability setup to start, but you do need signals. Track these from logs or simple dashboards:

• Spike in duplicate deliveries (often normal, but big jumps can signal timeouts or provider issues)
• High error rate by event type (for example, invoice payment failed)
• Growing backlog of events stuck in retry
• Mismatch checks (paid invoice but missing entitlement, revoked subscription but access still active)
• Sudden increase in processing time

If you’re building this in AppMaster, keep event storage in a dedicated table in the Data Designer and make “mark processed” a single, atomic decision point in your Business Process.

Next steps: test, monitor, and build it in a no code backend

Testing is where idempotency proves itself. Don’t only run the happy path. Replay the same event several times, send events out of order, and force timeouts so your provider retries. The second, third, and tenth delivery should change nothing.

Plan for backfilling early. Sooner or later you’ll want to reprocess past events after a bug fix, schema change, or provider incident. If your handler is truly idempotent, backfilling becomes “replay events through the same pipeline” without creating duplicates.

Support also needs a small runbook so issues don’t turn into guesswork:

• Find the event ID and check whether it’s recorded as processed.
• Check the invoice or subscription record and confirm the expected state and timestamps.
• Review the entitlement record (what access was granted, when, and why).
• If needed, re-run processing for that single event ID in a safe reprocess mode.
• If data is inconsistent, apply one corrective action and record it.

If you want to implement this without writing a lot of boilerplate, AppMaster (appmaster.io) lets you model the core tables and build the webhook flow in a visual Business Process, while still generating real source code for the backend.

Try building the webhook handler end to end in a no-code generated backend and keep it safe under retries before you scale traffic and revenue.