Stop Duplicate n8n Webhook Side Effects with an Idempotency Gate

Webhook retries are a reliability feature, not an edge case. A provider can deliver the same event again when an acknowledgement is lost, a network path fails, or your endpoint takes too long to respond. If an n8n workflow sends an email, creates a shipment, grants access, or posts a ledger entry every time the trigger fires, one logical event can produce several real-world side effects.

Before implementing the gate, use Webhook Retry Lab to compare duplicate actions, lost events, and recoverable retries. Move the simulated crash before or after the side effect to see why a seen flag and destination idempotency solve different problems.

The practical fix is an idempotency gate: verify the request, derive a stable event key, atomically claim that key in durable storage, and let only the successful claimant continue. This article builds that gate with n8n and PostgreSQL. It prevents concurrent duplicate executions from both passing the check, while keeping retry and recovery behavior explicit.

Why a normal lookup is not enough

A common n8n design looks sensible:

1. Search a data store for the webhook event ID.
2. If it is missing, perform the side effect.
3. Save the event ID as processed.

The race lives between steps 1 and 3. Suppose two retries arrive within milliseconds. Execution A looks up the ID and finds nothing. Before A saves it, execution B performs the same lookup and also finds nothing. Both branches continue. The workflow has a duplicate guard, but it is not atomic, so it still duplicates the side effect.

Stripe explicitly warns that webhook endpoints can receive the same event more than once and recommends recording processed event IDs. It also notes a more subtle case: two distinct Event objects can describe duplicate changes to the same underlying object. That distinction affects how you choose a key, which we will return to below.

The safe workflow shape

The gate should run near the start of the workflow, before any irreversible action:

1. Receive: accept the webhook and preserve the original request body.
2. Authenticate: verify the provider signature before trusting an event ID or payload field.
3. Normalize: extract a provider name, stable event ID, event type, and the fields required by downstream nodes.
4. Claim: insert the event key into PostgreSQL with a unique constraint.
5. Route: continue only if the insert returned a row; otherwise take the duplicate path.
6. Act: perform the business side effect with its own downstream idempotency protection where available.
7. Complete: mark the claim as completed and record useful outcome metadata.

Signature verification comes before the claim. For Stripe, verification must use the unmodified raw request body. If middleware or a Code node parses and reserializes the body first, verification can fail because the byte sequence changes. An unauthenticated sender must not be able to reserve arbitrary event IDs in your gate.

For a provider-neutral implementation of that first authentication step, see how to verify HMAC-SHA256 webhook signatures in n8n from the raw request body before the idempotency claim.

Create a durable claim table

Use a composite primary key so different providers can safely use overlapping ID formats:

CREATE TABLE webhook_event_claims (
  provider       text        NOT NULL,
  event_id       text        NOT NULL,
  event_type     text        NOT NULL,
  status         text        NOT NULL DEFAULT
                               'processing',
  first_seen_at  timestamptz NOT NULL DEFAULT now(),
  completed_at   timestamptz,
  last_error     text,
  attempt_count  integer     NOT NULL DEFAULT 1,
  PRIMARY KEY (provider, event_id)
);

The primary key is the concurrency control. PostgreSQL can resolve competing inserts against it atomically. The status fields are not required for basic deduplication, but they make failure recovery much safer than a permanent “seen” flag with no context.

Keep this table in a durable production database, not in workflow static data or process memory. Multiple n8n workers and executions must share the same authority. Back it up, restrict write permissions, and monitor its growth like any other operational table.

Claim the event in one PostgreSQL node

Add a PostgreSQL node after signature verification and normalization. Choose Execute Query and use this statement:

INSERT INTO webhook_event_claims (
  provider,
  event_id,
  event_type
)
VALUES ($1, $2, $3)
ON CONFLICT (provider, event_id)
DO NOTHING
RETURNING event_id;

Provide the three values through the node’s Query Parameters option rather than concatenating expressions into SQL. For example, pass normalized fields such as stripe, evt_..., and invoice.paid. n8n documents parameterized queries for the PostgreSQL node and sanitizes query parameters, which also keeps payload data out of SQL syntax.

The result is deliberately binary:

• One returned row: this execution created the claim and owns the right to proceed.
• No returned row: another execution already owns or completed the same key; do not run the side effect again.

PostgreSQL documents that ON CONFLICT DO NOTHING avoids inserting a row when the proposed row conflicts with a usable unique constraint. The RETURNING clause gives the workflow a direct signal from the same atomic statement. There is no vulnerable lookup-to-insert gap.

Route the n8n result without accidental fall-through

Connect the PostgreSQL node to an IF node that tests whether a returned item contains event_id. The true branch is the winner path. The false or empty-result path is the duplicate path. Depending on the PostgreSQL node version and output settings, a zero-row result may produce no item at all; test this behavior in your exact n8n version. If necessary, enable an output option that preserves an empty result or wrap the claim in a function that always returns an explicit claimed boolean.

The duplicate branch should avoid every non-idempotent downstream node. Return a successful response only when your state model says the original delivery has been accepted. A duplicate record with status='completed' can normally receive a 2xx acknowledgement immediately. A record still marked processing may also receive 2xx if an independent recovery process guarantees completion, but that choice is operational policy—not a universal rule.

Use n8n’s Respond to Webhook node for a controlled acknowledgement such as:

{
  "received": true,
  "duplicate": true
}

Do not include secrets or internal database details in the response. Most providers only need a timely success status.

Completion and failure semantics matter

An atomic claim prevents two executions from acting at the same time, but it does not create exactly-once delivery. Consider a crash after the claim is inserted but before the side effect finishes. Every retry now sees a duplicate claim. If the duplicate path always returns 2xx, the event can be lost permanently.

Model the claim as a small state machine:

• processing: one execution owns the event and work may be in progress.
• completed: the intended side effect finished successfully.
• failed: the attempt ended with a known error and is eligible for an explicit retry policy.

After the side effect succeeds, update the row:

UPDATE webhook_event_claims
SET status = 'completed',
    completed_at = now(),
    last_error = NULL
WHERE provider = $1
  AND event_id = $2;

On a controlled failure, record a sanitized error and move the row to failed. A separate recovery workflow can retry failed or stale processing rows after a lease interval. Do not simply delete the claim on every error: if the side effect actually succeeded but the acknowledgement or status update failed, deletion reopens the gate and can duplicate the action.

For high-value actions, add a fencing mechanism or transactional outbox. The ideal design commits the claim and an outbox message in one database transaction; a worker then delivers the side effect and records the result. When the downstream API supports an idempotency key, pass a stable key derived from the provider and event ID. A business-level unique constraint—such as one shipment per order or one ledger row per provider transaction—adds another essential layer.

Choose the right deduplication key

For a literal retry of the same Stripe Event object, event.id is the natural delivery key. Stripe defines the Event id as a unique identifier, and its webhook guidance recommends logging processed event IDs.

However, a provider can sometimes create two different Event objects for changes to the same resource. Stripe’s guidance suggests using the underlying data.object ID together with event.type to identify those cases. Treat that as a business deduplication decision, not a universal replacement for the delivery key. Two events of the same type for one object may represent distinct legitimate state transitions. Only collapse them when the provider’s semantics and your business rules prove they are equivalent.

A robust schema may therefore store both:

• a delivery key: (provider, event_id);
• an optional business key: for example (provider, event_type, object_id, business_version).

The delivery key protects against transport retries. The business key protects a specific domain invariant. Keeping those purposes separate makes incidents easier to reason about.

Retention, replay, and observability

Do not delete claims as soon as a provider stops its normal retry schedule. Manual replay, delayed queues, and support operations can occur later. Stripe, for example, documents automatic retries for live-mode webhook deliveries for up to three days, but your retention period should also cover manual replay and the cost of a duplicate side effect.

A practical starting point is 30 to 90 days for ordinary events and longer for financial or compliance-sensitive operations. Archive before deletion if you need an audit trail. Any cleanup query should target only old terminal rows, never active processing claims.

Track at least these metrics:

• claims created, grouped by provider and event type;
• duplicate claims rejected;
• time spent in processing;
• failed claims and retry outcomes;
• signature verification failures;
• claim-to-completion latency.

An unexpected duplicate spike can reveal provider retries, timeouts, a slow acknowledgement path, or an internal loop. A growing set of stale processing rows usually signals that recovery is missing or a downstream dependency is failing.

Test the gate before production

A single happy-path webhook proves almost nothing. Run a test matrix that exercises concurrency and failure boundaries:

1. Concurrent duplicate: send the same valid event ID to the test endpoint many times in parallel. Exactly one execution should reach the protected side effect.
2. Invalid signature: reuse the payload with a bad signature. It should fail before any claim is inserted.
3. Crash before side effect: stop the winner immediately after the claim. Confirm that your stale-claim recovery policy can resume safely.
4. Crash after side effect: stop it after the external action but before marking completion. Confirm that the downstream idempotency key or business constraint prevents a second action.
5. Same object, different event types: confirm that legitimate events are not collapsed by an over-broad business key.
6. Replay after retention: verify what happens when an old claim has been archived or deleted, and document that behavior.

Inspect the external system—not just n8n’s execution list—to verify that only one email, charge, shipment, entitlement, or ledger entry exists. A clean workflow log does not prove the side effect was unique.

Redis is an alternative, with the same atomicity requirement

Redis can implement a fast claim with SET key value NX EX seconds. Redis documents that NX sets the key only if it does not already exist, while EX applies an expiry. That can work well when the n8n integration exposes the required options or when you call a controlled Redis command path.

Do not translate it into separate GET and SET nodes; that recreates the race. Also choose the expiry carefully: once the key expires, an old replay can pass the gate again. PostgreSQL is often easier when you need durable states, audits, explicit recovery, and business constraints. Redis is attractive for high-throughput short-lived claims, but it does not remove the need for downstream idempotency or failure-aware state.

Production checklist

• Verify the webhook signature against the original request bytes.
• Normalize and validate the provider event ID and type.
• Use an atomic unique insert, never a lookup followed by an insert.
• Let only the returned-row branch reach side effects.
• Represent processing, completed, and failed explicitly.
• Use downstream idempotency keys or business unique constraints.
• Define stale-claim recovery and duplicate acknowledgement policy.
• Retain claims long enough for retries and manual replay.
• Load-test concurrent copies of the same event.
• Alert on stale processing rows and duplicate spikes.

The important guarantee is modest but powerful: for one event key, only one concurrent workflow execution wins the database claim. Combined with signature verification, durable state, downstream idempotency, and tested recovery, that gate turns duplicate webhook delivery from an incident into an expected code path.

Official references

• Stripe: Handle duplicate events
• Stripe: Resolve webhook signature verification errors
• Stripe API: The Event object
• n8n: PostgreSQL node documentation
• PostgreSQL: INSERT, ON CONFLICT, and RETURNING
• Redis: SET command