Agent Memory in Postgres: One DB, No Vector Store

You can store all four agent memory types, episodic, semantic, procedural, and working, in one Postgres using pgvector, JSONB, hybrid search with reciprocal rank fusion, and pg_cron for lifecycle, with no separate vector database and no sync layer.

Episodic events, semantic facts, procedural skills, and the working buffer have different lifecycles, but they share the same relational context: the user, the tenant, the session, the tool call that produced them. Splitting them across a separate store fractures that context and buys latency, ops surface, and a sync layer you didn't need. The rest of this piece walks through why the popular frameworks push toward external vector DBs anyway, what the four memory types actually require, and how the Postgres-native pattern holds up on LongMemEval and LoCoMo, the benchmarks that stress memory instead of one-shot RAG.

Why Mem0, LangMem, and Letta push you toward a separate vector DB

The short answer: each framework was designed as a memory layer meant to sit above whatever store you already had, so their reference architectures treat the vector index as an external component you plug in. A survey of Mem0, Zep, Letta, and Cognee makes the shape clear: Mem0 does extraction-based fact memory as a drop-in library, Zep/Graphiti builds a bi-temporal knowledge graph, Letta ships a full stateful-agent runtime with memory tiers, and Cognee is a pipeline that turns documents into a graph+vector store. Different tools, same architectural assumption: embeddings live somewhere that isn't your primary database.

The default stack: vector store + key-value layer

The canonical setup is Pinecone or Qdrant for embeddings, Redis or DynamoDB for session state, and Postgres for everything else the app already needed. Memories get written to the vector store, session summaries to the KV layer, and the agent stitches them together at query time. It works. It also means three consistency models, three failure domains, and three IAM stories.

The hidden tax of external vector databases

The pgvector vs Qdrant comparison, updated 2026-03-13, is worth reading in full. The punchline: benchmarks published in early 2026 show both systems delivering sub-100ms retrieval latency on standard agent-memory workloads, so the differentiator has shifted from raw speed to operational tradeoffs. When your memory sits in the same Postgres as your users, projects, and tool audit logs, a memory query is a JOIN. When it sits in Qdrant, it's a network hop, a schema drift risk, and a second bill.

That second bill is real. So is the drift. As Aquifer's authors put it, most AI memory systems bolt a vector DB on the side; the alternative is treating PostgreSQL as the memory itself. Sessions, summaries, turn-level embeddings, and entity graph all live in one database, queried with one connection. No sync layer, no eventual consistency, no separate vector database to keep aligned.

The four types of agent memory you actually need to store

The schema falls out of the taxonomy, so start there. A December 2025 write-up from Machine Learning Mastery by Vinod Chugani and Zylos's April 2026 architectures survey both frame long-term memory as three tiers, episodic, semantic, procedural, mirroring the cognitive-science distinction. Add the short-lived working buffer on top and you have the four things production agents need to persist.

Episodic memory: events and experiences

Episodic memory is the log of what happened. A user asked X, the agent called tool Y, the workflow failed on step 3. As AegisDB's design notes point out, episodic records are immutable; you append rather than rewrite. That makes them cheap to store and easy to reason about, but expensive to search naively, because there are a lot of them.

Semantic memory: durable facts and knowledge

Semantic memory is what the agent knows: the user prefers dark mode, the customer's contract renews in March, "ERR-4521" refers to a Postgres connection pool exhaustion event. Facts change. They get corrected, contradicted, and superseded. Semantic memory is where the bitemporal machinery pays for itself, because you can ask "what did we believe last Tuesday?" months later without losing today's corrections.

Procedural memory: learned skills and workflows

Procedural memory is how the agent does things: the prompt template that worked, the tool-call sequence that resolved a class of tickets, the reflection that "when the user says 'urgent', escalate before summarizing." Chugani's worked example frames it well: episodic memory recalls that last month's renewable-energy report followed a certain outline, while procedural memory is the outline itself as a reusable skill.

Working memory: the short-lived context buffer

Working memory is the current turn's scratchpad, tool results, partial reasoning, and the last few user messages the agent needs right now. It's volatile by definition. Powabase's agent runtime manages context automatically: before each LLM call we estimate token count and, if we're nearing the model's context limit, apply a sliding-window strategy that keeps the N most recent turns, then summarize the older conversation with a lightweight LLM if we're still over. Working memory only has to persist long enough to finish the turn.

Agent memory is not RAG, and that changes the storage design

Agent memory retrieves from a stream you're still writing; RAG retrieves from a static corpus you curated. That difference reshapes the storage design because the index optimal for one is wrong for the other.

The Dakera team makes this concrete: developers reach for a vector database because they equate memory with retrieval. But a vector database is a lookup index. It won't extract facts from a conversation or decay stale information on its own, and it doesn't know that "the user's manager" mentioned yesterday is the same entity as "Sarah" mentioned today. Ask it about Sarah, and it happily returns yesterday's chunk about a nameless "manager" with no idea the two refer to the same person; ask it about the manager, and it misses everything filed under Sarah.

The Vectorize team frames the upstream problem even more sharply: useful agent memory isn't a pile of conversation chunks, it's a structured representation that gets built and rebuilt as conversations happen. Their recommended default is a learning pipeline that writes facts, entities, edges, and synthesized observations into one Postgres with pgvector for embeddings. The pipeline, extract facts, link entities, resolve contradictions, synthesize observations, is what makes memory memory. A vector DB is one index over the output of that pipeline, not a substitute for it.

That's why the storage design pulls back toward a relational database. Facts have IDs, entities have edges, observations have provenance, and everything has a valid time window. Postgres already models all of that.

The Postgres-native pattern: one database, all four memory types

One memories table, one embedding column, JSONB for shape, an mtype tag to distinguish episodic from semantic from procedural, plus a companion sessions table for working memory. That's the whole architecture.

Projects like pgAgent, a PostgreSQL extension plus Python toolkit that stores memories, chunks, embeddings, categories, and importance in native tables, and kagent's memory backend, which requires Postgres with pgvector enabled, both organize around the same shape.

Schema design with pgvector, JSONB, and mtype tags

A minimal shape:

create extension if not exists vector;

create table memories (
 id bigserial primary key,
 tenant_id uuid not null,
 agent_id uuid not null,
 mtype text not null check (mtype in ('episodic','semantic','procedural')),
 content text not null,
 content_hash bytea not null,
 metadata jsonb not null default '{}'::jsonb,
 embedding vector(1536),
 tsv tsvector generated always as (to_tsvector('english', content)) stored,
 importance real not null default 0.5,
 valid_from timestamptz not null default now(),
 valid_to timestamptz,
 superseded_by bigint references memories(id),
 created_at timestamptz not null default now(),
 unique (tenant_id, agent_id, content_hash)
);

JSONB carries the shape that varies by mtype (tool call payloads for episodic, entity references for semantic, prompt templates and success rates for procedural) without forcing a rigid schema per type. Powabase uses the same pattern under the hood: our ai schema exposed via PostgREST manages agent runs, sessions, knowledge bases, and tool results as first-class Postgres rows, with JSONB for the payload variability and pgvector for the embeddings.

HNSW vs IVFFlat: choosing the index for agent memory

For agent memory, HNSW is the right default in pgvector. It gives better recall/latency tradeoffs on the small-to-medium collections most agents accumulate, and it handles the incremental inserts that memory writes generate, with no rebuild step and no training set. As Vectorize notes, the old "just use Postgres" objection was that pgvector's IVFFlat indexes weren't competitive at scale; HNSW closed that gap. IVFFlat is fine when you have tens of millions of vectors and can afford periodic re-clustering; agent memory rarely hits that scale per tenant.

create index memories_embedding_hnsw
 on memories using hnsw (embedding vector_cosine_ops);

create index memories_tsv_gin on memories using gin (tsv);
create index memories_tenant_agent on memories (tenant_id, agent_id, mtype);
create index memories_metadata_gin on memories using gin (metadata jsonb_path_ops);

Idempotent ingest with content hashing

Agents produce duplicates. A tool that runs three times with the same input shouldn't create three memories. The content_hash column plus the unique constraint on (tenant_id, agent_id, content_hash) makes writes idempotent; an INSERT... ON CONFLICT DO NOTHING collapses duplicates at the database, not in application code. This is the kind of correctness guarantee that's easy in one database and hard across a vector store plus a KV layer.

Hybrid retrieval inside Postgres (HNSW + BM25 + RRF)

Vector similarity alone loses to keyword search on certain queries and wins on others. The AegisDB team puts the problem crisply: embeddings average rare tokens away, so identifiers like --tenant-max-records or hnsw.c:214 can be unfindable by the exact string you remember. A BM25-style index keeps identifiers intact and finds them verbatim. Hybrid search with pgvector on one side and tsvector on the other, fused into a single ranked list, gets exact matches and topical ones both to surface.

Vector similarity with pgvector cosine distance

The dense side is a straight ORDER BY embedding <=> $1 LIMIT k. Cosine distance (<=>) is the standard for text embeddings; the HNSW index handles the search in sub-linear time.

Keyword search with tsvector and a GIN index

The lexical side is ts_rank(tsv, plainto_tsquery($1)) against the generated tsvector column, backed by the GIN index above. This is the same full-text search Postgres has shipped for years, no exotic extension, no extra process.

Fusing results with Reciprocal Rank Fusion (RRF)

The AgentOS Postgres backend docs document the exact query shape that's become the community default for RRF: a dense CTE, a lexical CTE, and a fusion CTE that merges them with 1/(k + rank_dense) + 1/(k + rank_lexical), followed by a final join to fetch full rows. k = 60 is the conventional constant. Aquifer ships the same RRF ranking pattern as a PG-native package: turn-level embedding, hybrid RRF, and an optional knowledge graph, all on PostgreSQL and pgvector.

with dense as (
 select id, row_number() over (order by embedding <=> $1) as r
 from memories
 where tenant_id = $2 and agent_id = $3
 order by embedding <=> $1 limit 50
),
lex as (
 select id, row_number() over (order by ts_rank(tsv, plainto_tsquery($4)) desc) as r
 from memories
 where tenant_id = $2 and agent_id = $3
 and tsv @@ plainto_tsquery($4)
 limit 50
),
fused as (
 select coalesce(d.id, l.id) as id,
 coalesce(1.0/(60 + d.r), 0) + coalesce(1.0/(60 + l.r), 0) as score
 from dense d full outer join lex l using (id)
)
select m.*, f.score
from fused f join memories m on m.id = f.id
order by f.score desc limit 10;

Purpose-built agent-memory systems like Dakera take this further. Their pipeline runs HNSW → top 50, BM25 → top 50, RRF → top 20, then a cross-encoder rerank → top 5. Dakera attributes the biggest recall lift on ambiguous conversational queries to the rerank stage specifically: RRF fuses two decent-but-noisy candidate lists, and the cross-encoder does the semantic tiebreak that neither dense nor lexical retrieval gets right alone. Every step of that pipeline is expressible in Postgres. The rerank stage can call an external model or a local cross-encoder; the rest is SQL. Powabase's retrieval strategies run this same stack, vector, BM25, hybrid, tree, with cross-encoder reranking on top, all against the project's own pgvector-backed Postgres.

Decay-aware recall: similarity × importance × recency

Memory decay matters for agent retrieval. A fact learned yesterday usually matters more than one from six months ago, and an "important" flag on a memory should bias retrieval toward it. Aquifer's 3-way hybrid retrieval adds a sigmoid time decay with configurable midpoint and steepness, plus an entity boost when sessions mention query-relevant entities. In SQL, the final ORDER BY becomes something like:

order by f.score
 * (0.5 + 0.5 * m.importance)
 * exp(-extract(epoch from now() - m.created_at) / (86400 * 30))
 desc

Thirty-day half-life, importance weight, RRF score. Tune the constants per workload.

Managing the memory lifecycle without extra infrastructure

Summarization, consolidation, supersession, and expiry all run inside Postgres itself, pg_cron schedules the jobs, valid_from/valid_to columns handle bitemporal history, and a superseded_by self-reference records contradiction resolution. No background worker fleet required.

Scheduled summarization and consolidation with pg_cron

pg_cron runs SQL on a schedule. That's enough to periodically collapse episodic runs older than N days into semantic summaries, roll up entity mentions into an entity table, and expire working-memory rows past their TTL. A nightly job that reads yesterday's episodic memories for each agent, calls an LLM to synthesize durable facts, and writes them back with mtype = 'semantic' is a single SELECT... FROM... WHERE created_at > now() - interval '1 day' plus a function call.

Bitemporal memory with valid_from / valid_to

Facts have two clocks: when they were true in the world, and when we knew them. Bitemporal modeling with valid_from and valid_to columns lets you answer both "what did we believe last Tuesday?" and "what was actually true last Tuesday?", which is critical for auditing agent decisions after the fact. The Zep/Graphiti line of work is built entirely around this; you get the same guarantees in vanilla Postgres with two timestamp columns and discipline.

Supersession and contradiction resolution

When a new fact contradicts an old one, don't delete the old one. Set its valid_to = now() and point superseded_by at the new row. Queries that want "current truth" filter on valid_to is null; queries that want history don't. Immutable episodic memory and updatable semantic memory can share a table because supersession lives in a column rather than requiring a separate schema.

Multi-tenant memory isolation with Row-Level Security

If your agents serve multiple users or customers, memory isolation is not optional. A leaked semantic memory is a leaked fact about someone else's business. Postgres RLS solves this at the database. Every query gets rewritten to include the tenant predicate, so an application bug can't accidentally cross the boundary.

Powabase's RLS model is our reference for the pattern: distinct Postgres roles for anon, authenticated, and service_role, JWTs signed by the platform, and default policies that deny by default on user tables. On the memories table, the policy is one line: using (tenant_id = auth.jwt() ->> 'tenant_id'). A separate vector database would need its own tenant model, its own auth, and its own audit trail, three more places for the isolation to leak.

When a dedicated vector database still makes sense

Being honest about where Postgres stops being the right answer matters. The pgvector vs Qdrant analysis puts the boundary well: start with pgvector for architectural simplicity and relational data governance, and move to a dedicated vector engine when retrieval latency at the p95-p99 tail, hybrid sparse+dense search at very large scale, or subagent memory isolation becomes a first-class operational concern.

Concretely: you have hundreds of millions of vectors per tenant, you need sub-20ms p99 across that corpus, or you're running a specialized ANN workload (multi-vector, ColBERT-style late interaction) that pgvector doesn't yet support well. For those cases, Qdrant, Weaviate, or Pinecone earn the extra ops cost. For most agent memory Postgres workloads, they don't.

How Postgres memory holds up on LongMemEval and LoCoMo

The doubt worth taking seriously is whether one-database memory can actually match purpose-built systems on quality, not just simplicity. The benchmarks say yes. The AgentOS team's Postgres backend reports 85.6% on LongMemEval-S (1.4 points above Mastra at gpt-4o) and 70.2% on LongMemEval-M, using pgvector, HNSW, tsvector, and RRF, exactly the stack described above. AgentOS 0.3.0+ runs the entire cognitive Brain on Postgres, not just the vector store, and those scores are on the same infrastructure.

LongMemEval stresses long-horizon recall across sessions; LoCoMo stresses conversational memory over months of simulated dialogue. What both benchmarks reward is the learning pipeline upstream of retrieval, the hybrid search, and the decay and importance signals at rank time, not raw ANN throughput. All of that lives above the storage layer, and all of it composes cleanly on Postgres.

Agent memory is about modeling relationships between facts, entities, and time. Once you accept that, one Postgres with pgvector, JSONB, tsvector, RRF, RLS, and pg_cron covers episodic, semantic, procedural, and working memory with fewer moving parts and better isolation than the default stack. Powabase gives you a per-project Postgres with the retrieval, reranking, and agent runtime already co-located, the same pattern this article describes, without the assembly. Start with the schema above, benchmark against LongMemEval on your own traces, and add a dedicated vector engine only if the p99 numbers tell you to.