RAG Backend, Not Framework: Agentic Loops on Postgres

Agentic RAG only looks like a framework problem. Wire up LangGraph, add a grader node, loop back on failure, and the diagram fits on a napkin. The reason production teams stall isn't the graph. Every node in the loop hammers the retrieval layer with a different pattern: filtered lookups, hybrid recall, rewritten follow-ups, cache probes, loop-state reads and writes. If your retrieval layer is a bolt-on vector service sitting next to your database, the loop pays the tax on every hop.

A working RAG backend collapses those calls into one Postgres: vectors, BM25, ACLs, loop state, and cache in the same transactional store the rest of the app already uses. This piece walks the Decide→Grade→Re-Query loop node by node and shows what each one actually asks of the substrate underneath.

Why agentic RAG's real dependency is a backend, not a framework

Plain RAG is a pipeline: embed, retrieve top-k, generate, return. Every query takes the same path, and the model never decides anything about retrieval. It consumes whatever comes back. That's fine until retrieval is wrong, at which point the LLM produces a confident answer from bad evidence and you have no way to notice.

Agentic RAG replaces the straight line with a control loop. A planner decides what to retrieve, a grader evaluates what came back, and a rewriter reformulates when the grade is poor. The framing shifts from "what chunks match this query" to "what information do I need to answer this, and which tools can supply it". Frameworks like LangGraph model this cleanly as nodes and conditional edges.

As fast.io's write-up on agentic RAG puts it, autonomous agents dynamically decide what to retrieve, when to retrieve it, and how to use retrieved information, which means each iteration of the loop issues fresh retrieval calls with different filters, different query strings, and different intents. It also has to remember what it already tried so the next hop doesn't repeat itself. A framework wired to a remote vector database and a separate Postgres for state ends up doing a distributed transaction on every step. A RAG backend that keeps vectors, lexical search, ACLs, and loop state in one database turns each hop into local SQL.

The Decide→Grade→Re-Query loop in one page

The loop has three moving parts. Decide routes the query. Is this a factual lookup, a comparison, an aggregation? Does it need retrieval at all? Grade inspects the retrieved chunks (and often the draft answer) for relevance and groundedness. Re-Query kicks in when the grade fails: rewrite the query, widen filters, fall back to a different tool, and go again, up to a hard cap.

A concrete self-correcting implementation looks like this in the planner contract:

const MAX_HOPS = 3;
const MIN_CONFIDENCE = 0.6;

type Hop = { query: string; chunks: RetrievedChunk[]; confidence: number };

The planner returns one of retrieve (with a refined sub-query) or answer, and the loop halts on confidence or hop count. Every hop appends to loop state; every decision reads it.

CRAG (Corrective RAG) vs. Self-RAG vs. Adaptive-RAG

These are three shapes of the same loop.

• CRAG (Corrective RAG) grades retrieval and, on failure, corrects, usually by rewriting the query or falling back to web search before generation.
• Self-RAG grades the generated answer against the retrieved context (groundedness) and triggers a rewrite if the answer isn't supported. Ungrounded answers trigger a query rewrite, not a retry with the same context.
• Adaptive-RAG routes at the top: cheap queries get a single pass, complex ones enter the loop. The router prevents you from paying loop cost on questions that don't need it.

Real systems combine all three: an adaptive router in front, CRAG-style retrieval grading inside, and a Self-RAG groundedness check before returning. The reflection agent sits between retrieval and generation with the authority to send the loop back to retrieval with a modified query rather than proceeding to generation, and generation only fires once the accumulated context grades as sufficient.

Where LangGraph fits, and where the framework stops

LangGraph is a good fit for the control-flow layer. Its agentic RAG tutorial models the loop as a graph: start with generate_query_or_respond, route on whether the model made tool calls, grade retrieved document content, then generate or rewrite. Nodes, edges, conditional routing. That's what a graph framework is for.

What the framework does not give you is retrieval. retriever_tool is a stub you fill in with your own query against your own store. If that store is a managed vector DB, every graph node crosses a network boundary. If it's Postgres with pgvector next to your application tables, the graph node is a function call.

What each loop node actually demands from retrieval

The loop's real weight lands on retrieval, and each node asks for something different.

Decide/route: metadata filters and query classification

Routing needs cheap, deterministic pre-filtering: tenant, document type, date range, ACL scope. A query classified as "billing policy for enterprise tier, EU region" should retrieve only from documents that match those attributes. In a specialised vector DB, filtering by tenant, date, or category usually turns into payload filtering with its own recall cliffs. In Postgres, it's WHERE tenant_id = $1 AND doc_type = 'policy' on indexed columns, composed with the vector search.

Grade: hybrid recall so the grader has something worth grading

A grader can only mark chunks it actually sees. If dense retrieval misses the passage that contains the answer, the grader will correctly say "not relevant" and the loop will re-query, burning tokens on a recall problem the retriever should have solved.

The critic pattern is three sequential grades: retrieval relevance, answer groundedness, and answer utility, each with a conditional edge that can reroute or halt. All three depend on wide, precise recall on the first pass. Hybrid search, BM25 for exact terms and vectors for semantics, fused together, is the practical way to give the grader that.

Re-query: query rewriting plus loop state to avoid repeats

The rewriter needs to know two things: what the user actually asked, and what the loop has already tried. Without that history, a rewriter will happily generate a paraphrase that returns the same chunks and fails the same grade. Loop state (prior queries, prior chunk IDs, prior confidence scores) belongs somewhere the next hop can read cheaply. Not in memory that dies with the process. Not across a network to a state service. In the same database as the vectors.

The governed retrieval substrate on one Postgres

Teams keep landing on the same answer. Put all of it in Postgres. Vectors, tsvector, ACL columns, session and loop tables, semantic cache. One backup, one connection pool, one transaction boundary. RAG grounds LLM responses in specific, up-to-date knowledge by retrieving semantically relevant passages at query time and injecting them into the prompt as context, and the store that holds those passages is the same store the rest of the app already writes to.

Dense retrieval with pgvector and HNSW tuning

pgvector with an HNSW index is the reasonable default for dense retrieval. Up to roughly 10M vectors per node it holds without a second stack or a second backup plan; pgvectorscale with StreamingDiskANN pushes the ceiling to ~50M with p95 under 50ms.

The tuning that matters in an agent loop is hnsw.ef_search. Set it high for the first retrieval hop where recall is critical (the grader needs the right chunks in-scope), and consider a lower value for cache probes where you want speed. Because it's a session GUC, the loop can set it per node.

Hybrid search: BM25 (tsvector) + vectors fused with RRF

Dense-only retrieval misses exact-string matches like SKUs, error codes, version numbers, and proper nouns. Postgres's tsvector handles the lexical side natively, and Reciprocal Rank Fusion (RRF) combines the two rankings without needing calibrated scores. The pattern shows up cleanly as SQL:

-- Step 1: Query Rewriting
rewritten_query := rag.rewrite_query(question);
-- Step 2: Hybrid Search
SELECT string_agg(content, E'\n---\n') INTO context
FROM rag.hybrid_search(rewritten_query, 5);

That's the Tencent Cloud reference for a Postgres-native agentic RAG function, and it's the same shape you end up with anywhere: rewrite, hybrid-search, generate, all as SQL against one connection. The Tencent reference decomposes it further into a Planner Agent, Retriever Agent, and Generator Agent, all running against the same Postgres.

Metadata filters and document ACLs without the recall cliff

Multi-tenant RAG lives or dies on ACLs. If the wrong document reaches the LLM, that's a data leak. Postgres row-level security enforces it at the query layer. The same policies that protect your relational tables protect the embeddings sitting in a vector column next to them. In Powabase, our RLS model runs the retriever under the caller's identity, so the retriever sees only rows the caller is allowed to see. No parallel permissions system for the vector store.

Loop state and semantic cache in the same database

The loop needs a table (or a couple) to track hops: query text, filters used, chunk IDs returned, grade, timestamp. A semantic cache (embedding of the question → cached answer) is another vector index. Both are ordinary Postgres. The re-query node writing loop state and the router node reading the cache happen in the same transaction as the retrieval that just ran.

Preventing infinite loops: max retries, escape hatches, and the decision rule

An agentic loop without hard stops is a bill generator. Two guards are non-negotiable: a max-iteration cap and a doom-loop detector. The n8n guidance is blunt. Hard caps on iterations and a token budget keep a reasoning loop from spinning and burning cost when it can't converge.

The ergini production checklist is even more specific: a hard max-hop budget of 3 to 5 hops enforced in code, not a prompt instruction the model might ignore, and a falling confidence threshold per hop that forces commitment rather than endless retries. Powabase's ReAct runtime enforces that cap in code and refuses to expose tools on the final hop, so the model is forced to produce a text answer instead of another tool call. For the "keeps rewriting to the same thing" failure mode, we guard on progress rather than string equality. A min_new_chunks_per_iteration check fails a hop that returns zero new chunks against loop state, catching the paraphrase-that-changes-nothing case before it eats a budget.

The decision rule for exiting the loop wants three conditions ORed together: confidence ≥ threshold, hop count ≥ max, or no-progress detected. When any fires, generate a best-effort answer from what you have and mark the response as low-confidence for the caller.

Measuring the loop: retrieval quality and decision accuracy

If you can't measure the loop, you can't tell whether it's earning its cost. Two axes to instrument.

Retrieval quality per hop. Recall@k and precision@k on the graded chunks, tracked separately for hop 1, hop 2, hop 3. If hop-1 recall is already 0.9, most queries shouldn't reach hop 2 — a router problem. If hop-3 recall is barely above hop-1, rewriting isn't helping, which is a rewriter or filter problem.

Decision accuracy. How often does the grader agree with a human label? A grader that says "irrelevant" too readily forces needless re-queries; one that's too lenient lets hallucinations through. Sample and label a few hundred hops.

Both need trace data (the query, the filters, the chunks, the scores, the grade) persisted per hop. Our sessions API returns a run trace like { retrieved_context: [{ id, score, retrieval_score, reranker_score, source_name, included_in_context }] }, giving you the fields to compute the metrics after the fact rather than trying to reconstruct them from logs.

Latency and cost trade-offs of adding the agentic loop

The loop is not free. A single-pass RAG call is one embedding, one retrieval, one generation. A three-hop CRAG call is up to three retrievals, three grader calls (small model), one or two rewrites, and one final generation. Agentic RAG runs roughly 5 to 50× the cost of plain RAG, and the reflection pattern alone tends to double latency and quadruple cost.

Two mitigations that actually move the number:

1. Route before you loop. Adaptive-RAG's classifier decides whether to enter the loop at all. Simple lookups skip it entirely.
2. Cheap grader, expensive answerer. The dbi-services reference uses gpt-5-mini for the decision hops and gpt-5 for the final synthesis. The grader runs many times; the answerer runs once. Splitting them by model class is the single largest cost lever.

Semantic caching cuts the tail further (a hit skips the loop entirely) and belongs in the same Postgres that holds the vectors.

Postgres vs. a dedicated vector database for a RAG backend

Pinecone, Qdrant, and Weaviate solve one problem well: fast, large-scale vector search. For an agentic loop, that's not the only problem.

The operational cost of a separate vector service is real. A second backup regime, a second scaling plan, and a synchronization layer between your source-of-truth database and the vector store. Most teams reach for a dedicated vector database — Pinecone, Weaviate, and the rest — by default, then discover that every insert, update, and delete now has to reach two systems and stay consistent between them.

The query-shape cost is worse. Metadata filtering in a specialised vector DB is payload filtering with its own recall behaviour; joining the retrieved chunks against tenant tables, document tables, or ACL tables means round-tripping IDs back to Postgres. In pgvector, it's a join.

The honest counter is that at scales past ~50M vectors per node, or extreme QPS with strict p50 SLOs, purpose-built engines win on raw vector throughput. For the vast majority of production agentic RAG (mid-scale corpora, hybrid queries, per-tenant ACLs, a loop that touches retrieval three times per user question) Postgres is faster end-to-end because the loop isn't paying network tax on every hop.

How Powabase runs the grade/re-query cycle from one Postgres

Powabase is the RAG backend under this loop. Every project gets its own isolated Postgres, Realtime, and Storage, with retrieval, rerank, and the agent runtime co-located so RAG stays hot and agent loops stay short. vector and pg_net are preloaded at provision time; pg_trgm and friends are one CREATE EXTENSION away.

Retrieval is first-class: five indexing strategies (ChunkEmbed, Full Document, PageIndex, GraphIndex, Doc2JSON) and four retrieval methods (vector, BM25, hybrid, tree) with reranking on top. Hybrid is the default recommendation for general RAG. The retrieval settings (HYBRID_DEFAULT_VECTOR_WEIGHT, KB_DEFAULT_TOP_K, reranker model) are all configurable per project.

The agent runtime handles the loop side. Our ReAct loop enforces the safeguards described above: a hard hop cap in the 3–5 range, no-progress detection on iterations that return no new chunks, tools withheld on the final hop, and retry-with-continuation on truncated outputs. Sessions persist the full retrieval trace (chunks, scores, reranker scores, whether each was included in context) as structured JSON, so evaluating retrieval quality per hop is a query, not a log-scraping project. When one agent isn't enough, our Supervisor orchestration lets a coordinator delegate to entity agents that each have their own tools and knowledge bases, still on the same Postgres.

Costs work the way the model above suggests. web_search runs at $0.020–0.040 per call depending on tier, LLM inference is billed by whichever provider key you bring, and the platform base itself starts at $0 with a per-project stack included. You pay for the calls the loop actually makes.

When to reach for the agentic loop, and when plain RAG wins

The loop earns its cost on questions single-pass RAG demonstrably fails: comparisons across topics, multi-hop reasoning, queries with ambiguous intent, and anything where the right retrieval requires reading a first result before knowing what to ask next. The dbi-services example, "Compare PostgreSQL and MySQL indexing approaches": the agent searches one, detects the gap, searches the other, then synthesizes, is the archetype. Single-pass would mix both into noisy context and miss the differences.

Plain RAG wins when the query is a factual lookup against a well-chunked corpus, when latency matters more than the last 10% of quality, or when your evaluation shows hop-1 recall is already high enough. Don't loop for the sake of looping. Instrument first, and let the router decide per query.

Build it on a database that already holds your vectors, your BM25 index, your ACLs, and your loop state, and the loop becomes a few SQL calls and a small model deciding what to do next. Adaptive router in front, CRAG-style retrieval grading in the middle, Self-RAG groundedness check at the end, 3–5 hops around the whole thing, and a trace table you can actually query afterward.