Long-Running Agents on Postgres: Durable, Resumable State

An agent that plans across hours or days will crash. The container gets recycled, a deploy rolls out on a Tuesday afternoon, an OOM kills the worker, and the in-memory plan — partial results, the "where was I" pointer, tool outputs waiting to be reconciled — evaporates all at once. For a chat turn you retry. For a job that's been running six hours, retry is the wrong verb.

Put the state in Postgres, make compute disposable, and design the schema so a fresh process can pick up mid-graph without double-firing a tool call. This piece walks through the concrete session and checkpoint model behind durable long-running agents: how thread_id ties invocations together, how LangGraph's PostgresSaver writes per-node checkpoints, how idempotency keys keep retries safe, and how SELECT FOR UPDATE SKIP LOCKED turns a Postgres table into a durable job queue for crash recovery. We ship the same model at Powabase under ai.agent_sessions, and it's what makes eval replay and time-travel debugging possible after the fact.

Why long-running agents die on plain in-memory state

An in-memory session dict lives in the process and dies with it. Every rolling deploy, every scale-out event, every worker OOM is a mass amnesia event for whatever conversations that worker was holding. That's fine for a stateless chat endpoint. It's a data-loss bug for an agent that's been researching a customer's contract for the last forty minutes.

Zylos names four distinct failure scenarios, each demanding a different recovery strategy:

A single "just persist stuff" answer doesn't cover all four. What does cover them is treating every state transition as an ACID write to Postgres, so every transition is persisted and every failure recoverable, with decisions like "which tool was picked at step 7" still queryable a week later.

The design question is the smallest durable record that lets a fresh process finish the job.

The one move: make Postgres the source of truth

Pick one durable store and put everything through it. Don't scatter it across Redis for messages, DynamoDB for state, and S3 for artifacts — put it in one Postgres, and your durability ceiling becomes the database's, not any given worker's uptime.

Why a separate checkpoint store or Redis is the trap

The tempting design keeps the plan in Redis or an in-process state machine "for speed" and flushes to Postgres later. That works until the process dies mid-run and the plan, partial results, and pointer all vanish together. Splitting durability across two stores splits your transaction boundary: a message can be acknowledged in Redis while the corresponding step write to Postgres never lands, and you can't tell after the fact which side won.

A cleaner rule from Clausey: transitions update, retries append. One row per grain (run, step-attempt, agent, trace) and every write goes through the same database that answers your queries later. Zylos sketches a three-layer production architecture that follows the same rule: execution durable in LangGraph or DBOS with Postgres checkpointing, messages in a redelivery-safe queue, structured memory in the same DB.

What Powabase's native Postgres gives you

Powabase ships the full agent stack on real open-source Postgres. Our ai.agent_sessions table is first-class: every long-running agent session is a multi-turn conversation record owned by the caller, persisting across runs until explicitly deleted, carrying message history, retrieved context per assistant turn, and per-run reasoning configuration.

Because it all lives in one schema, you can join session state to your application tables in a single query, back it up in a single pg_dump, and reason about it with one set of RLS policies. For a broader tour of how memory sits in that same Postgres, see our writeup on agent memory in Postgres without a separate vector store.

The session/state model: thread_id as the conversation key

Under the durability rule, the schema question becomes: what grain do you write at? Answering that is what separates a schema you can reason about from a JSON blob you'll regret.

agent_sessions, session_messages, and session_events tables

Three tables carry the load:

• agent_sessions, one row per multi-turn conversation: thread_id as the primary key, owning user, created_at, status, current checkpoint pointer, and a small JSONB for run configuration.
• session_messages, append-only, each with thread_id, a monotonic sequence number, and a role: user inputs, assistant responses, tool calls, tool results.
• session_events, the fine-grained event stream (start, step_started, tool_call, complete), with timestamps, so you can reconstruct exactly what happened when.

Keeping the three grains separate matters. Messages are the model's view of the conversation. Events are the operator's view of execution. Sessions are the product's view of "this thing the user is doing." Trying to serve all three from one wide table is where schemas devolve into metadata JSONB mush.

thread_id as the primary key that ties every invocation together

The thread_id is the join key for everything downstream. In LangGraph's PostgresSaver, it's the config field that identifies which conversation a checkpoint belongs to: {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}. In the general Postgres-session pattern, the session ID is a client-supplied UUID that the client saves after the first call and passes on every subsequent one. In your own tables, use it as the foreign key from messages, events, tool runs, and checkpoints. That single key is what makes a conversation a recoverable unit.

Per-node checkpoints with LangGraph PostgresSaver

Session-level state answers "what's the conversation?" It doesn't answer "which node in the graph were we executing when the process died?" That's what per-node checkpointing is for.

How PostgresSaver checkpoints each super-step

LangGraph's PostgresSaver writes a checkpoint after every super-step in the graph. Attach it at compile time and the graph resumes from the last completed node when you re-invoke with the same thread_id:

from langgraph.checkpoint.postgres import PostgresSaver

DB_URI = "postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable"
with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
    checkpointer.setup()
    graph = builder.compile(checkpointer=checkpointer)
    config = {"configurable": {"thread_id": job_id}}
    state = graph.get_state(config)

The row-key design is worth internalizing:

CREATE TABLE checkpoints (
  thread_id TEXT NOT NULL,
  checkpoint_ns TEXT NOT NULL DEFAULT '',
  checkpoint_id TEXT NOT NULL,      -- ULID, lexicographically sortable
  parent_checkpoint_id TEXT,
  type TEXT,
  checkpoint BYTEA,
  metadata JSONB,
  PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
);

The checkpoint_id is a ULID, so "give me the latest state for this thread" is an index-only descending scan and "give me the state at time T" is a bounded range query. Both matter later, for replay.

StateSnapshot fields and what a checkpoint actually stores

A checkpoint isn't just the graph's state dict. It's a StateSnapshot: the values at that node, the next node(s) to execute, the configurable identifiers (thread_id, checkpoint_ns, checkpoint_id), a metadata blob (source, step number, writes), a parent pointer for the causal chain, and the pending writes that were staged but not yet committed to the next node. That parent pointer is what lets you walk the history like a git log, and it's what makes branching from a past checkpoint straightforward.

Durability modes: sync, async, and exit

Not every checkpoint needs an fsync before the next node runs. LangGraph exposes three durability modes:

Pick the mode that matches how expensive re-running the last node would be.

Write safety: idempotency keys and duplicate-write prevention

Durable state is only half the problem. The other half is what happens when a retry re-executes a node whose external side effect already landed — the payment went through, the email was sent, the row was inserted — but the checkpoint write didn't. Without idempotency, retry becomes double-fire.

Deterministic hashing and the unique idempotency_key index

The pattern is a deterministic hash over the operation's identifying fields, stored as a column with a unique index. The generic Postgres-session pattern uses a hash of (session ID + role + content + turn number) so a retried write with the same key hits the unique constraint and returns the original row instead of inserting a duplicate. Catch the violation, read back the original, and move on.

For any external write in an agent step, hash the tool-call fields — say sha256(thread_id + step_id + user_id + amount) for a payment — and pass that key to the downstream API (or check-and-insert in your own DB) before doing the work. A retry re-derives the same key and no-ops.

Retry-safe appends after a database timeout

The awkward case is a write that times out and you don't know if it landed. The append-only messages pattern handles this cleanly: load the session before the LLM call, write after, and use an idempotency key on the write. If the write succeeded but the ack didn't return, the retry hits the unique constraint, you interpret that as "already applied," and move on. If the write actually failed, the retry succeeds. Either way, no duplicate row.

Crash recovery: detecting and resuming a dead agent process

You have durable state and safe writes. What tells a fresh worker that thread 42 was mid-run when its old worker died?

PENDING status, leases, and fencing tokens

The AXME model is a clean sketch: a status column that moves through CREATED -> SUBMITTED -> DELIVERED -> IN_PROGRESS -> COMPLETED. When the agent crashes at IN_PROGRESS, the row stays at IN_PROGRESS. No timer, no cron, just durable state. A restarted agent calls listen() and the framework redelivers the intent up to max_delivery_attempts. AXME layers this under LangGraph, CrewAI, and other frameworks by replacing their in-process saver with its intent lifecycle.

To make that safe under concurrent workers, add two columns: a lease_expires_at timestamp and a lease_owner worker id. A worker claims a job by setting both under a transaction; the lease renews while work progresses; if the worker dies, the lease expires and another worker can claim it. A monotonically increasing fencing_token on each claim prevents a zombie worker from committing writes after its lease expired: the current token in the row won't match its stale one, and the write is rejected.

SELECT FOR UPDATE SKIP LOCKED as a durable job queue

The claim itself is a Postgres one-liner:

UPDATE agent_jobs
SET status = 'IN_PROGRESS',
    lease_owner = $1,
    lease_expires_at = now() + interval '60 seconds',
    fencing_token = fencing_token + 1
WHERE job_id = (
  SELECT job_id FROM agent_jobs
  WHERE status = 'PENDING'
     OR (status = 'IN_PROGRESS' AND lease_expires_at < now())
  ORDER BY created_at
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
RETURNING *;

The combination of SELECT ... FOR UPDATE SKIP LOCKED, a lease column, and a reclaim cron is a complete durable queue. No Redis, no SQS, no separate broker. Multiple workers can poll this same table concurrently and never step on each other.

Human-in-the-loop resume from a checkpoint

Sometimes long-running agents shouldn't recover automatically. They should stop, wait for a human to approve a purchase or edit a draft, and resume from exactly where they paused. That's a checkpoint operation, not a new problem.

The pattern: the graph reaches a node that emits a pending_approval event and writes a checkpoint. The status column moves to AWAITING_HUMAN. The API returns the checkpoint_id and the staged action to the UI. When the human approves (or edits), the frontend POSTs the decision with the thread_id and checkpoint_id, and the backend calls graph.invoke(input, config) with that config. LangGraph loads the StateSnapshot, applies the human's input as the next value, and continues from the pending writes that were staged before the pause.

Because the checkpoint is the resume point, "the human took three days to click approve" is indistinguishable from "the worker crashed for three days." Both are just gaps between checkpoint write and next invocation.

Replaying checkpoints as eval rows

The same schema that recovers crashed agents also gives you free evals. Every checkpoint is a labeled point-in-time snapshot. Every session is a completed trajectory. You already have the ground truth; you just have to replay it.

Time-travel from a checkpoint_id to reconstruct a session

The parent pointer on every checkpoint means the full causal chain of a session is a recursive CTE away. Pick any checkpoint_id and you can walk backward to the initial state, or forward through descendants to see everything that happened next. Because checkpoint_id is a lexicographically sortable ULID, "give me every checkpoint for this thread between T1 and T2" is a bounded index range scan.

To replay: fetch the checkpoint at the point you want to fork from, feed it back through graph.invoke with a new model or a modified prompt, and compare the new trajectory to the recorded one. Nothing about this requires a separate eval framework. It's the same PostgresSaver read path you already trust for crash recovery.

Turning replayed sessions into ground-truth eval comparisons

The recorded messages, tool calls, and tool results become the reference trajectory. Replay the same input under a new model or system prompt and diff the outputs: did the same tool get called with the same arguments? Did the final assistant message reach the same conclusion? For structured outputs, string equality or a schema check gets you 80% of the way. For open-ended text, an LLM judge scores replay vs. ground truth.

Managing long sessions: context compaction and retention

A session that runs for weeks eventually blows through the model's context window. Durable state doesn't help you here. You can persist an infinite conversation, but you can't feed one to the LLM.

When to compact a long-running session's context window

Compaction summarizes older messages so recent ones can stay verbatim. The trigger question is: at what fraction of the context budget do you compact? Too aggressive and you summarize away detail that mattered. Too lazy and you'll hit the limit mid-tool-call and blow up a run. A reasonable default: compact when projected input tokens exceed 70% of the model's window, keep the last N=20 messages verbatim, and store the summary as a new "system" message at the head of the trimmed history. The original messages stay in session_messages; compaction changes what you send to the model, not what you keep on disk.

Archiving and pruning old sessions without losing audit data

Retention is a separate axis from context management. Sessions that finished six months ago probably don't need to be in the hot table; they still need to be queryable for audit and eval. Move them: a nightly job copies sessions with completed_at < now() - interval '90 days' into an archive schema, drops them from the hot tables, and keeps the archive on cheaper storage. If you need to replay one for an eval, you copy it back; if you need to answer a compliance question, you query the archive directly. Never DELETE without a copy. An archived session is still a labeled trajectory you'll want later.

How this compares: LangGraph vs DBOS vs bolt-on stores

Three broad approaches show up in production:

Powabase's position for long-running agents is that most teams don't want to pick. LangChain and LangGraph are powerful abstractions, but they're frameworks, not infrastructure. You write the code, you deploy the checkpointer, you run the Postgres, you wire up the queue, you build the eval replay tooling. We ship the session table, the message history, the streaming persistence, idempotency-keyed billing, and RLS policies, so you can drop into the same schema and run your own recursive CTE against ai.agent_sessions when you want to. Under the hood it's the same pattern this whole article describes: durable Postgres state, keyed by thread_id, tracing a session from start to complete.

The specific detail that changes the day-to-day is having the session grain built in. You don't decide whether to model agent_sessions; it's already there, indexed on user_id, joinable to your tables. What you build on top is the rest of your product.

The payoff is concrete: one Postgres, one schema, one transaction boundary. Get that right and a mid-afternoon deploy stops taking your six-hour research jobs with it. The user's next message lands on a fresh worker, the last checkpoint loads, and the agent picks up at the node it was on.