The Transactional Outbox Pattern: Reliable Events Without Distributed Transactions

A service that saves an order and then announces that order is doing two writes to two different systems: a row into PostgreSQL and a message onto a broker. Neither write is hard on its own. The problem is that they fail independently, and a failure in between leaves the two systems permanently disagreeing about what happened. The transactional outbox pattern is the standard fix: write the event into an outbox table inside the same database transaction as the business data, and let a separate dispatcher publish it afterwards.

The guarantee is easy to state: an event is published if and only if the transaction that produced it commits. No distributed transaction, no coordinator spanning two systems - just the database transaction you were already running.

sequenceDiagram
    autonumber
    participant App as Service
    participant DB as Database
    participant D as Dispatcher
    participant MQ as Message broker
    App->>DB: BEGIN
    App->>DB: INSERT order + outbox row
    App->>DB: COMMIT
    Note over DB,D: the event is now as durable as the data
    D->>DB: claim unpublished rows (SKIP LOCKED)
    DB-->>D: batch of events
    D->>MQ: publish
    D->>DB: mark row published

This post builds the pattern end to end with Go and PostgreSQL: the table, the writer, the dispatcher, and the operational details that most explanations leave out - retries, poison messages, ordering, latency and cleanup.

The dual-write problem

The naive sequence looks harmless: update the database, commit, then publish the event. Two things can go wrong, and both are bad in different ways.

sequenceDiagram
    autonumber
    participant App as Service
    participant DB as Database
    participant MQ as Message broker
    App->>DB: BEGIN
    App->>DB: INSERT order
    App->>DB: COMMIT
    Note over App,MQ: the process dies here
    App--xMQ: publish OrderCreated - never happens

If the process dies, is restarted mid-deploy, or loses the network after the commit but before the publish, the order is saved and nobody is ever told. The reverse failure is worse: the publish lands first and the transaction then rolls back, so downstream systems react to an order that does not exist. AWS Prescriptive Guidance calls this the dual write operation problem: “A failure in one of these operations might result in inconsistent data.”

Retries do not fix it. If the publish after the commit fails, a retry needs to know whether the failure happened before or after the broker received the message - and the process that could answer is the one that just died. The window between commit and publish is small, but a busy service crosses it millions of times a day, and it is crossed by exactly the events that matter: payments, signups, shipments.

Why not a distributed transaction

A two-phase commit (2PC) spanning the database and the broker would make both writes atomic. In practice it is rarely the right tool, and the canonical description of the pattern gives the reasons: one or both systems may not support it at all, the service becomes coupled to the availability of both, and a 2PC coordinator that stalls holds locks in two systems instead of one. Most message brokers do not speak the XA protocol that database drivers use for this, and bolting it on costs throughput where you can least afford it.

The outbox pattern gets the same atomicity from a plainer observation: the database transaction is already atomic, so put the event inside it.

The outbox table

An outbox row is the event, stored where your data lives:

create table outbox_event (
    id              bigint generated always as identity primary key,
    aggregate_type  text        not null,
    aggregate_id    text        not null,
    event_type      text        not null,
    payload         jsonb       not null,
    created_at      timestamptz not null default now(),
    published_at    timestamptz,
    attempts        int         not null default 0,
    next_attempt_at timestamptz not null default now(),
    leased_until    timestamptz
);

create index outbox_event_due
    on outbox_event (next_attempt_at, id)
    where published_at is null;

A few deliberate choices:

• The payload holds the final event body, not a pointer. The outbox row is the event of record. A dispatcher should never have to re-read business tables, re-derive state, or reconstruct what the event meant - the event you publish is byte-for-byte the event you wrote, even if it goes out minutes later.
• The aggregate columns say what the event is about. They drive routing and ordering, which turns out to be the hardest part of the pattern.
• published_at is null means undelivered. The attempts, next_attempt_at and leased_until columns are the dispatcher’s bookkeeping; the partial index keeps the hot lookup - “what is due?” - scanning only undelivered rows.

The table lives in the same database as the business data. That is the entire trick: the transaction you were already committing becomes the atomic unit that covers the event too.

Writing the event inside the transaction

The writer changes one thing: the event insert joins the transaction it belongs to.

package main

import (
	"context"
	"encoding/json"
	"strconv"

	"github.com/jackc/pgx/v5/pgxpool"
)

type Order struct {
	CustomerID int64
	TotalCents int64
}

func CreateOrder(ctx context.Context, pool *pgxpool.Pool, order Order) (int64, error) {
	tx, err := pool.Begin(ctx)
	if err != nil {
		return 0, err
	}
	defer tx.Rollback(ctx) // no-op if Commit succeeds

	var orderID int64
	err = tx.QueryRow(ctx,
		`insert into orders (customer_id, total_cents, status)
		 values ($1, $2, 'created')
		 returning id`, order.CustomerID, order.TotalCents).Scan(&orderID)
	if err != nil {
		return 0, err
	}

	payload, err := json.Marshal(map[string]any{
		"orderId":    orderID,
		"customerId": order.CustomerID,
		"totalCents": order.TotalCents,
	})
	if err != nil {
		return 0, err
	}

	var outboxID int64
	err = tx.QueryRow(ctx,
		`insert into outbox_event (aggregate_type, aggregate_id, event_type, payload)
		 values ('order', $1, 'order.created', $2)
		 returning id`,
		strconv.FormatInt(orderID, 10), payload).Scan(&outboxID)
	if err != nil {
		return 0, err
	}

	// Wake the dispatcher on commit; see "Waking the dispatcher" below.
	_, err = tx.Exec(ctx, `select pg_notify('outbox_event', $1)`,
		strconv.FormatInt(outboxID, 10))
	if err != nil {
		return 0, err
	}

	return orderID, tx.Commit(ctx)
}

If the transaction commits, both rows exist. If anything fails - a constraint, a crash, a lost connection - neither does. There is no window left between the two writes, not a small one: the database’s own atomicity is the guarantee. One thing to decide early is the payload’s shape and stability, because the event may be published long after the transaction that wrote it, and consumers will decode it without access to the code that produced it.

Dispatching: claim, publish, mark

The dispatcher is a background loop: claim a small batch of due rows, publish each one, mark it published. The claim is a single PostgreSQL statement:

with candidates as (
    select id
    from outbox_event
    where published_at is null
      and next_attempt_at <= now()
      and (leased_until is null or leased_until < now())
      and attempts < 10
    order by id
    limit 100
    for update skip locked
), claimed as (
    update outbox_event e
    set attempts      = e.attempts + 1,
        leased_until  = now() + interval '30 seconds'
    from candidates c
    where e.id = c.id
    returning e.id, e.aggregate_type, e.aggregate_id,
              e.event_type, e.payload, e.attempts
)
select * from claimed order by id;

for update skip locked is what lets several dispatchers share one table without colliding: rows another dispatcher holds are skipped rather than waited on. The PostgreSQL documentation describes exactly this use: “Skipping locked rows provides an inconsistent view of the data, so this is not suitable for general purpose work, but can be used to avoid lock contention with multiple consumers accessing a queue-like table.” The clause has existed since PostgreSQL 9.5, whose release notes introduce it plainly: “Add SELECT option SKIP LOCKED to skip locked rows.”

Three deliberate choices in that statement:

Locks are released before publishing. The claim transaction commits as soon as the update returns; the row locks never span the network call to the broker. Holding them across a slow publish would stall every other dispatcher and anything else contending for those rows. Instead the claim sets a short lease in leased_until: if a dispatcher dies mid-publish, the lease expires and the row becomes claimable again. That behaviour is a feature, and the next section is about what it costs.

Rows are locked in id order. Because the query scans in order by id, concurrent dispatchers acquire row locks in the same order, which is what keeps them from deadlocking each other.

The batch comes back in id order, but only because the outer query says so. RETURNING emits rows in whatever order the update produced them; nothing carries the CTE’s order by through to the result. The select ... from claimed order by id wrapper is what actually makes the batch ordered, and the ordering section below leans on it.

The Go side of the loop is small:

package main

import (
	"context"
	"errors"
	"log/slog"
	"os"
	"os/signal"
	"syscall"
	"time"

	"github.com/jackc/pgx/v5/pgxpool"
)

const claimBatch = `
with candidates as (
    select id
    from outbox_event
    where published_at is null
      and next_attempt_at <= now()
      and (leased_until is null or leased_until < now())
      and attempts < 10
    order by id
    limit $1
    for update skip locked
), claimed as (
    update outbox_event e
    set attempts = e.attempts + 1,
        leased_until = now() + interval '30 seconds'
    from candidates c
    where e.id = c.id
    returning e.id, e.event_type, e.aggregate_id, e.payload, e.attempts
)
select id, event_type, aggregate_id, payload, attempts
from claimed order by id
`

type OutboxEvent struct {
	ID          int64
	EventType   string
	AggregateID string
	Payload     []byte
	Attempts    int
}

func claim(ctx context.Context, pool *pgxpool.Pool, limit int) ([]OutboxEvent, error) {
	rows, err := pool.Query(ctx, claimBatch, limit)
	if err != nil {
		return nil, err
	}
	defer rows.Close()

	var events []OutboxEvent
	for rows.Next() {
		var e OutboxEvent
		if err := rows.Scan(&e.ID, &e.EventType, &e.AggregateID, &e.Payload, &e.Attempts); err != nil {
			return nil, err
		}
		events = append(events, e)
	}
	return events, rows.Err()
}

func publish(ctx context.Context, e OutboxEvent) error {
	// Hand the event to the broker here. Returning an error marks the
	// attempt failed and schedules a retry.
	return nil
}

func markPublished(ctx context.Context, pool *pgxpool.Pool, id int64) error {
	_, err := pool.Exec(ctx,
		`update outbox_event set published_at = now() where id = $1`, id)
	return err
}

func scheduleRetry(ctx context.Context, pool *pgxpool.Pool, e OutboxEvent) error {
	_, err := pool.Exec(ctx,
		`update outbox_event
		 set next_attempt_at = now() + (interval '5 seconds' * pow(2, $1))
		 where id = $2 and published_at is null`, e.Attempts, e.ID)
	return err
}

func RunDispatcher(ctx context.Context, pool *pgxpool.Pool) {
	ticker := time.NewTicker(500 * time.Millisecond)
	defer ticker.Stop()

	for {
		select {
		case <-ctx.Done():
			return
		case <-ticker.C:
			batch, err := claim(ctx, pool, 100)
			if err != nil {
				if !errors.Is(err, context.Canceled) {
					slog.Error("outbox: claim failed", "err", err)
				}
				continue
			}
			for _, e := range batch {
				if err := publish(ctx, e); err != nil {
					slog.Error("outbox: publish failed, will retry",
						"id", e.ID, "attempt", e.Attempts, "err", err)
					if err := scheduleRetry(ctx, pool, e); err != nil {
						slog.Error("outbox: scheduling retry failed", "id", e.ID, "err", err)
					}
					continue
				}
				if err := markPublished(ctx, pool, e.ID); err != nil {
					// The message may already be out; the row stays
					// unpublished and the lease decides when it is retried.
					slog.Error("outbox: marking published failed", "id", e.ID, "err", err)
				}
			}
		}
	}
}

func main() {
	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()

	pool, err := pgxpool.New(ctx, os.Getenv("DATABASE_URL"))
	if err != nil {
		slog.Error("outbox: connect failed", "err", err)
		os.Exit(1)
	}
	defer pool.Close()

	RunDispatcher(ctx, pool)
}

The failure paths are the interesting part:

flowchart TD
    A["poll: any rows due?"] --> B["claim a batch with SKIP LOCKED"]
    B --> C["publish each event"]
    C -- "success" --> D["mark the row published"]
    C -- "failure" --> E["attempts + 1, schedule retry with backoff"]
    E --> F{"attempts over the cap?"}
    F -- "yes" --> G["stop retrying, alert a human"]
    F -- "no" --> A
    D --> A

A failed publish bumps attempts and pushes next_attempt_at out exponentially, so a struggling broker gets breathing room instead of a hammering. When attempts reaches the cap in the claim query, the row stops being claimed at all - that is the poison-message guard. Without it, one undeliverable event sits at the front of the id order and is retried forever, in front of every healthy event queued behind it. A capped row needs a human or a dead-letter path, which is why the cap should come with an alert, not just a number.

A dispatcher is a long-running service like any other, so it should shut down gracefully: the context cancellation in the loop above stops claiming new batches, lets the in-flight batch finish, and exits - the same drain-and-exit shape as an HTTP server.

At-least-once delivery and idempotent consumers

Look at the crash windows in the loop and one fact falls out: the same event can be published twice. The dispatcher can die after the broker accepted the message but before markPublished committed; the lease expires; another dispatcher republishes. A broker client timeout is the same event in disguise - the publish may have succeeded and the client given up waiting. The pattern’s canonical description says it directly: “The Message relay might publish a message more than once.”

So the delivery guarantee is at-least-once, and consumers must be idempotent. Two ways to get there, and they combine:

• Dedupe on the event id. Publish the outbox row’s id inside the message. A consumer keeps a table of processed ids with a unique constraint, so recording “seen” and applying the effect either both happen or neither does.
• Make the effect idempotent by construction. State transitions that are no-ops when repeated - “set status to paid where status is pending” - survive any number of replays without bookkeeping.

What is not on the menu is exactly-once end to end. Any design that publishes over a network and records progress in a database has the same windows; the outbox pattern just makes them visible and bounded instead of hidden.

Ordering is the hard part

The canonical pattern description adds a requirement that is easy to miss: “Messages must be sent to the message broker in the order they were sent by the service,” and that ordering “must be preserved across multiple service instances that update the same aggregate.” A customer who changes their plan twice should produce two events, and the second should not overtake the first.

Polling with skip locked fights that in two ways:

• Dispatchers run in parallel. Two batches claimed by two dispatchers can publish out of id order, and there is nothing in the claim statement to prevent it.
• The visibility gap. Identity values are assigned at insert time, but a row becomes visible only when its transaction commits. Transaction B writing event 101 can commit while transaction A writing event 100 is still in flight; a dispatcher polling in that instant sees only 101 and publishes it. When 100 finally commits, it lands after 101 even though it “happened first.”

Mitigations, in ascending strength:

1. A small dispatch delay. Add and created_at < now() - interval '2 seconds' to the claim. Most in-flight transactions commit within that window, so the visibility gap shrinks from an everyday event to a rare one. The cost is flat, honest latency.
2. Serialize per aggregate. Order within a batch is already preserved (that outer order by id), and routing by aggregate_id - broker partitions keyed on it, one consumer per partition - keeps one aggregate’s events on one path. Combined with the dispatch delay this covers the per-aggregate ordering that almost everyone actually needs.
3. Log-based dispatch (CDC). A change-data-capture tool reads the database’s transaction log, and PostgreSQL documents logical decoding as replaying changes “in the order they were made on the origin server.” Debezium’s outbox event router turns each captured outbox row into a message routed by aggregate - no polling, no visibility gap, because the log is the authority on order. This is the version to pick when ordering is a hard product requirement, at the price of running the CDC infrastructure.

Most systems discover that strict global ordering was never the requirement; per-aggregate ordering was. Knowing which one you need decides how far down this list you have to go.

Waking the dispatcher with LISTEN and NOTIFY

A 500ms poll scans a partial index that is usually empty, which is cheap - but the tail latency is one tick, and at high event rates the empty scans add up. PostgreSQL has a native doorbell. Notifications are transactional: the documentation states that “if a NOTIFY is executed inside a transaction, the notify events are not delivered until and unless the transaction is committed,” and that “if the transaction is aborted, all the commands within it have had no effect, including NOTIFY.” So one line in the writer’s transaction wakes the dispatcher exactly when the row becomes visible, and rolled-back work wakes nobody.

The same documentation draws the boundaries: the payload must stay under 8000 bytes (send the row id, not the event), identical notifications inside one transaction collapse into one, and the notification queue is bounded. Notifications are also transient - one that arrives while the dispatcher is reconnecting is simply gone. So NOTIFY is a hint, never the transport: the dispatcher keeps its slow poll as the safety net, and the notification only cuts the tail from “next tick” to “now.” The writer above already carries the one line it takes - the pg_notify call just before the commit, passing the outbox row’s id as the payload.

Cleaning the table up

A published row is history. Leave it and the table grows forever; the partial index stays fast, because it only ever indexed the undelivered rows, but the table underneath it keeps every delivered one - so autovacuum has more to do on each pass, the heap keeps growing on disk, and anything that does scan the table sequentially pays for rows that are only history. Two options: delete at publish time, or keep published rows for a retention window and delete them in batches. The second is usually better - the window doubles as a debugging aid (when a consumer swears it never received an event, the row is still there to prove otherwise), and the delete stays boring:

delete from outbox_event
where id in (
    select id from outbox_event
    where published_at is not null
      and published_at < now() - interval '7 days'
    limit 10000
);

Batched because a single delete of millions of rows holds locks for a long time, bloats the table and lags replicas; a scheduled delete of ten thousand is invisible. This is also a high-churn table, so it is worth checking that autovacuum is keeping up once the pattern is live at volume.

When not to use it

The pattern buys exactly one thing: atomicity between the database and the outside world. If nothing outside the database cares about the change, skip it - the transaction is already atomic, and the outbox would be an extra insert and an extra moving part for nothing.

If the event is fully derivable from the business tables, change data capture on those tables directly removes the double write too. The trade is intent: an outbox row is an explicit, versioned “this happened, in this shape” statement written by the code that knows, while events inferred from table diffs are coupled to whatever the schema happens to look like. Schema drift ages badly.

Event sourcing is the bigger sibling - the log of events becomes the state itself, and the outbox problem dissolves into it. That is more power and much more machinery; adopt it for the reasons event sourcing exists, not for the delivery guarantee alone.

Named honestly, the costs are small but real: every business write gains a second insert, the dispatcher is one more service to run and monitor, and delivery is asynchronous - a consumer sees each event at least one tick later. For most services that is a bargain next to the alternative, which is two systems that disagree about what happened, silently and forever.