
The n8n Error Handling Playbook: Retries, DLQs, Circuit Breakers, and Replay
The full production playbook for n8n reliability: how to classify failures, design retry policies that don't melt upstreams, build a Postgres DLQ with safe replay, run circuit breakers per integration, and wire the observability that catches incidents before your customers do.
n8n reliability is a design problem, not a settings toggle. Classify every failure as transient, semantic, or systemic; retry transient with jittered exponential backoff; DLQ semantic to a Postgres table with idempotent replay; open a circuit breaker for systemic outages. Wrap it with a single dashboard on success rate, DLQ depth, and breaker state — and you eliminate 95% of the incidents that page you today.
- Every failure is transient, semantic, or systemic. Same response = data loss.
- Full-jitter exponential backoff (500ms base, 30s cap, 5 attempts) is the safe default for most SaaS APIs.
- A DLQ is not a table — it's a table + replayer + dedup key + owner. Missing any of the four means the queue silently grows.
- Circuit breakers per integration cut cascading failures and reduce provider-side rate-limit bans.
- One dashboard: success rate, p50/p95 duration, DLQ depth, breaker state. Alert on rates of change, not absolutes.
The three failure modes — and why they can't share a code path
Every workflow you'll ever build in n8n fails in exactly one of three ways: transient (network blip, 429, timeout, brief provider hiccup), semantic (bad input, missing record, schema drift, business-rule violation), or systemic (auth expired, quota exhausted, provider outage, database down). Treating them the same is the number-one cause of silent data loss and pager fatigue in n8n shops.
| Failure | Right response | Wrong response | Cost of the mistake |
|---|---|---|---|
| Transient | Retry with jittered backoff | Fail fast → data loss | Executions dropped, customers see empty state |
| Semantic | Route to DLQ, never retry | Retry loop | Duplicate charges, spam, poisoned downstream systems |
| Systemic | Open a breaker, page a human | Retry harder | Full provider ban, hours of downtime |
The rest of this playbook shows exactly how to implement each response inside n8n — with copy-paste patterns, Postgres schemas, and the exact dashboards to build.
Classifying an error at the seam
Classification has to happen at the boundary between a workflow and the outside world — HTTP nodes, database nodes, LLM providers. Do it in a shared sub-workflow so every integration classifies the same way.
// classify.js — reusable Code node
const err = $json.error ?? {};
const status = err.status ?? err.statusCode ?? 0;
const message = (err.message ?? "").toLowerCase();
function classify() {
if ([408, 425, 429, 500, 502, 503, 504].includes(status)) return "transient";
if (status === 401 || status === 403) return "systemic"; // auth broken
if (status === 402 || /quota|rate limit exceeded/i.test(message)) return "systemic";
if (status >= 400 && status < 500) return "semantic";
if (/econnreset|etimedout|socket hang up/i.test(message)) return "transient";
return "systemic";
}
return [{ json: { ...$json, failureClass: classify() } }];Route the output on failureClass into three branches: retry, dlq, breaker. Every workflow in the org uses this classifier — no branch, no invention.
Retry policy that doesn't melt the upstream
n8n's built-in retry is fine for a few attempts on internal calls, but for anything customer-facing you want full-jitter exponential backoff. The math matters: without jitter, N clients that failed at the same time all retry at the same time and DDoS the recovering service.
// retry-delay.js — Code node inside a retry sub-workflow
const attempt = $json.attempt ?? 0;
const maxAttempts = 5;
const baseMs = 500;
const capMs = 30_000;
if (attempt >= maxAttempts) {
return [{ json: { ...$json, giveUp: true } }]; // -> route to DLQ
}
// AWS-style full jitter
const upper = Math.min(capMs, baseMs * 2 ** attempt);
const delayMs = Math.floor(Math.random() * upper);
return [{ json: { ...$json, attempt: attempt + 1, delayMs } }];Feed delayMs into a Wait node, then loop back into the flaky call. On giveUp, drop the payload into the DLQ described below. For payment providers, cap at 3 attempts and coordinate with their idempotency headers — retrying a charge without idempotency is how you double-bill customers.
Idempotency-Key. If in doubt, downgrade to a DLQ + human review — a duplicate charge is worse than a missed one.The Dead Letter Queue: a table is not enough
Most teams create a dlq table, dump failures, and call it done. Six months later it has 400,000 rows, nobody knows what's replayable, and half the entries reference schemas that no longer exist. A real DLQ has four parts: schema, owner, replayer, and dedup key.
create table workflow_dlq (
id uuid primary key default gen_random_uuid(),
workflow_id text not null,
execution_id text not null,
dedup_key text not null,
payload jsonb not null,
error jsonb not null,
failure_class text not null check (failure_class in ('transient','semantic','systemic')),
attempts int not null default 0,
owner_team text not null,
created_at timestamptz not null default now(),
replayed_at timestamptz,
resolved_at timestamptz,
unique (workflow_id, dedup_key) -- prevents floods on retry loops
);
create index on workflow_dlq (workflow_id, resolved_at) where resolved_at is null;The dedup_key is what makes the DLQ safe. Pick something idempotent — order ID + step name, message ID + handler, webhook delivery UUID. Without it, a broken workflow that retries in a tight loop will insert 10k rows in an hour and mask the real errors.
The owner_team column matters more than it looks. On day one, every DLQ row must have a human who owns it. Without an owner, the queue becomes a graveyard nobody reviews.
Replay: the workflow you'll build once and use forever
The DLQ replayer is a small, boring, extremely important workflow. It runs on a schedule, pulls unresolved rows, re-enqueues them, and marks results. Because dedup_key is unique, replaying the same row twice is a no-op.
- Cron trigger every 5 minutes → select up to 100 unresolved rows for the current workflow, oldest first.
- For each row: increment
attempts, call the target sub-workflow with the original payload. - On success → set
resolved_at = now(). On failure → updateerrorand re-classify. - If
attempts > 10, stop retrying automatically and open a Linear/Jira ticket assigned toowner_team.
Expose a manual replay button in an internal admin tool (a tiny Next.js or n8n Webhook page) so on-call engineers can flush a batch after an upstream recovers, instead of waiting for the next cron tick.
Circuit breakers per integration
When an upstream is down, retrying makes it worse: you burn your quota, extend the outage, and often trip the provider's abuse detection. A circuit breaker fixes this by refusing to call a known-dead service for T seconds.
Store the breaker in Redis (or a Postgres table with a for update skip locked read) as three states — closed, open, half-open:
- closed: requests pass through; consecutive failures increment a counter. At 5 failures in 60s, flip to
open. - open: every request short-circuits into the DLQ for T seconds (start at 30s, exponentially double on repeated flips).
- half-open: after T, allow exactly one probe. Success →
closed. Failure → back toopenwith double T.
// breaker.js — Code node called before every risky HTTP request
const svc = $json.service;
const key = 'breaker:' + svc;
const state = await redis.hgetall(key);
const now = Date.now();
if (state.status === 'open' && Number(state.openedUntil) > now) {
return [{ json: { ...$json, breaker: 'short-circuit' } }]; // -> DLQ
}
if (state.status === 'open') {
await redis.hset(key, { status: 'half-open' });
}
return [{ json: { ...$json, breaker: 'pass' } }];Idempotency: the second half of retries nobody talks about
Retries are only safe when the target operation is idempotent. That means the same call, made N times, has the same effect as one call. Most APIs support this through an idempotency key header — use it or design around it.
| Operation | Idempotency mechanism | n8n implementation |
|---|---|---|
| Stripe charge | Idempotency-Key header | Set from execution_id + step in every HTTP node |
| Postgres insert | Natural unique key + on conflict do nothing | Add unique index; treat conflict as success |
| Slack message | None natively | Store a sent flag in a dedup table before the send |
| Webhook to partner | Their signed message ID | Deduplicate on their ID in your inbox table |
For anything that touches money, users, or external systems, idempotency is not optional. Bake it in on day one — retrofitting is orders of magnitude more painful.
Observability you'll actually use
The goal is one dashboard that answers three questions in five seconds: is anything broken right now?, is anything drifting?, where do I look next?
- Send
execution.finishedevents from n8n to your logs pipeline (Loki, Datadog, BigQuery). - Tag every log line with
workflow_id,execution_id,failure_class, and a business correlation ID (order ID, user ID). - Build a single Grafana/Metabase dashboard: success rate per workflow, p50/p95 duration, DLQ depth per owner, breaker state per integration.
- Alert on rates of change, not absolutes — a 3× jump in DLQ depth in 10 minutes beats a fixed threshold at 500 rows.
- Wire alerts to the same channel where the owner team already lives (a Slack per team, not one #alerts firehose).
Testing the unhappy path before it tests you
You cannot claim reliability if you've never simulated failure. Bake three failure drills into your CI or a weekly Cron:
- Chaos node — a Code node that randomly throws with probability P. Turn P up to 1 in staging and confirm the DLQ fills correctly and the breaker opens.
- Latency injection — a Wait node with a random 0–30s delay in front of every external call. Confirms timeouts and downstream Wait/backoff paths.
- Provider outage — point a specific integration at a URL that returns 503. Confirm the breaker opens, the DLQ receives typed rows, and the replayer flushes on recovery.
None of these need to be fancy. Ten minutes of chaos in staging on Friday saves ten hours of forensic archaeology on Monday.
Common anti-patterns, and what to do instead
| Anti-pattern | Symptom | Fix |
|---|---|---|
| Retry-until-success on everything | Duplicate charges, spam | Classify first, retry only transient |
| Silent catch-all Try/Catch | Errors disappear, users see empty state | Rethrow into classify + DLQ path |
| DLQ without a replayer | Table grows to millions of rows | Ship replay workflow on day one |
DLQ without dedup_key | Retry loop floods the queue | Unique key per (workflow, business ID) |
| One #alerts channel for the whole org | Nobody reads it | Route per owner_team |
| Retrying auth failures | Provider bans your IP | 401/403 → systemic, open breaker |
| No idempotency key on Stripe/Adyen | Random double charges | Add key on day one |
A 90-minute checklist to harden any existing workflow
- Wrap external calls in the shared classify sub-workflow.
- Add the
workflow_dlqtable and route semantic + give-up cases into it. - Ship the replayer workflow with the 5-minute cron.
- Introduce the breaker sub-workflow in front of every third-party integration.
- Add
Idempotency-Keyheaders to every money- or user-touching HTTP call. - Emit
execution.finishedevents to your logs pipeline with rich tags. - Build the four-tile dashboard (success rate, p95, DLQ depth, breaker state).
- Run a chaos drill: crank the classify test path to 50% failure, confirm nothing leaks.
- Assign an
owner_teamto every workflow and route its alerts to that team's Slack. - Document the runbook in one Notion / Confluence page linked from the dashboard.
Frequently asked
Should I use n8n's Error Trigger or a custom DLQ?
Both. The Error Trigger is perfect for catch-all logging and alerting so nothing is silently swallowed. A custom DLQ table on top gives you replay, deduplication, ownership, and reporting — the four things Error Trigger alone can't do.
How aggressive can retries be before a provider notices?
For most SaaS APIs, 5 attempts with full jitter capped at 30s is safe. For payment providers, cap at 3 and always send an Idempotency-Key. For LLM providers, respect Retry-After headers and back off exponentially — they will rate-limit your whole account if you don't.
Where should the DLQ live — Postgres, Redis, or a queue like SQS?
Postgres, in 90% of cases. It gives you SQL for replay logic, transactional writes, unique constraints for dedup, and it's already in your stack. Move to SQS/Kafka only when volume exceeds ~1M rows/day or when you need cross-region fan-out.
How do I stop the DLQ from growing forever?
Two mechanisms. First, a hard cap on <code>attempts</code> (10 is a good default) after which the replayer stops and opens a ticket. Second, a monthly job that archives resolved rows older than 90 days into cold storage and deletes them from the hot table.
Do I need a breaker for internal services too?
Usually not. Breakers pay off when the cost of a failed call is high (rate limits, per-request billing, cascading auth revocation). Internal services usually just need a timeout + retry. Add a breaker only after an incident proves it's needed.
Isn't all this overkill for a small team?
The classifier + DLQ + replayer is ~2 days of work and eliminates the entire category of silent-loss incidents. Skip it and you'll spend 2 days per quarter on postmortems instead. It pays for itself in the first outage.
Building something similar?
I help teams ship production-grade AI agents, n8n workflows, and data platforms. Let's talk about what you're building.
Work with me
Islam Gamal
AI, Data & Automation Engineer. I design and ship production AI agents, n8n workflows, and cloud data platforms — with a focus on reliability, cost, and measurable business impact. Founder of Tashghil and Tek bil Arabi.
More from Islam Gamal
Building Production AI Agents with n8n: Architecture, Guardrails, Evals, and Cost Control
The full n8n agent playbook — reference architecture, memory design, tool routing, JSON-schema guardrails, retries, DLQs, evals, observability, and the small handful of decisions that keep the token bill sane at scale.
Webhooks That Don't Lie: Idempotency, Signatures, Replay, and a Playbook
An opinionated production guide to receiving webhooks — signature verification, idempotency keys, store-then-process pipelines, dead-letter queues, replay UIs, provider quirks, and a local testing loop your team will actually use.
Workflow Automation Blueprints: 12 Patterns Every Team Ships (and the Envelope That Ties Them Together)
The 12 automation blueprints I reuse across every client — lead routing, invoice processing, content ops, incident response — with the exact triggers, guards, envelopes, DLQs, and human-in-the-loop patterns that make them survive production.
Browse every article by Islam Gamal on the author page.