--- title: "Reference data's RAG economics — what the evidence licenses · Agentic Atlas" description: "Reference data's RAG economics — what the evidence licenses: The node carries no numeric floor (ruled 2026-08-02)." canonical: "https://agentic-atlas.dev/decisions/0014-reference-data-rag-economics" last-updated: "2026-08-19" --- Manifest-selected Decision Canonical id 0014-reference-data-rag-economics Status accepted Catalog order 7 Catalog revision `35263c4c415da742953d0462804fb14424e2244dae4c63efd27e468988de70ab` # 0014: Reference data's RAG economics — what the evidence licenses **Status:** accepted **Date:** 2026-08-03 ## Context `reference-data` argues that a corpus belongs on disk and enters the window as a query-shaped slice, and it argues where that move becomes RAG. Both arguments reach for numbers — retrieval failure rates, per-query bills, a corpus size at which frontloading is cheaper — and the field supplies numbers of wildly unequal grade. A research pass ran 2026-08-02 against the primary sources, and returned three things that could not all be carried at once: one vendor's measured ablation, one vendor's recommended threshold, and a first-party decision whose own author grades it as vibes. The pressure is that a pattern node is read as settled. A threshold quoted in it becomes a rule someone designs against; a rate quoted in it becomes the rate. This record fixes which of those the evidence licenses. ## Method and evidence grades Sources fetched directly: Anthropic's contextual-retrieval and context-engineering posts, Anthropic's embeddings and pricing docs, Lewis et al. 2020, Liu et al. 2023, and the OpenAI, Voyage and Pinecone pricing pages. Read at abstract level only: the 2026 agentic-retrieval benchmarks. Read secondhand: a practitioner interview write-up, a vendor counter-post, a debate round-up. Four grades result, and the node is bound to respect the difference: - **Measured and published with method.** Anthropic's retrieval ablation: top-20 retrieval failure **5.7%** at baseline, cut 35% by contextual embeddings (→3.7%), 49% by adding contextual BM25 (→2.9%), 67% by adding reranking (→**1.9%**). One vendor, one stack, one evaluation — measured, not general. Liu et al.'s U-curve is the other measured result carried: recall highest at the edges of a long context, "significantly degrades" in the middle, *even for explicitly long-context models*. - **Vendor-published prices, checked 2026-08-02.** Facts about a bill, not about behaviour, and true until the page changes. - **First-party assertion, self-graded.** Claude Code shipped with RAG and dropped it for agentic search — the decision and its stated reasons (simplicity, security/privacy, staleness, reliability) are beyond dispute; the *outperformance* is self-described as internal benchmarks "plus vibes." No published Anthropic benchmark of the comparison exists. - **Directional only.** The 2026 benchmarks at abstract depth; a vendor's 40%-fewer-tokens claim with a commercial stake and no methodology; a secondhand account of an internal study at another company. These characterize a failure regime. They do not measure one. One figure was **refused**: a 55.2%-vs-44.0% grep-vs-vector result circulating in search summaries could not be confirmed against the fetched abstract, and is carried nowhere. ## Decision **1 · The node carries no numeric floor** *(ruled 2026-08-02)*. The candidate was first-party and specific: below ~200,000 tokens (about 500 pages), include the whole knowledge base in the prompt, with prompt caching at **0.1× base input** absorbing the dollars. It is rejected as a threshold on two grounds — the source bills resident tokens, so the guidance is vendor-interested rather than neutral ground truth; and the figure predates a tokenizer change costing ~30% more tokens for the same text, per the pricing page's own note. Frontloading-with-caching is carried as a **competing posture, named and priced** — for a small, cold, high-fire-rate corpus it genuinely competes. What the node does not do is quote a size. "Too big to frontload" is qualitative: the corpus that visibly cannot fit, or that degrades the window when it does. **2 · Query breadth is its own crossover dimension** *(ruled 2026-08-02)*, and does not fold into vocabulary match. Grep holds when you know what you are looking for; **a broad search is a flood** — an exploratory query turns each pass into a noise dump regardless of whether the words match. Vocabulary bounds *reachability*; breadth bounds *admissible signal per search turn*. A targeted query with the wrong vocabulary is unreachable; a broad query with the right vocabulary is a flood, and the two need different fixes. **3 · BM25's seat belongs to the measured stack, not to RAG at large** *(ruled 2026-08-03; ablation reading corrected 2026-08-06)*. Contextual embeddings produced the largest incremental reduction in Anthropic's ablation (5.7%→3.7%); adding contextual BM25 improved it further (3.7%→2.9%), and reranking reduced it again (2.9%→1.9%). The result supports grep's statistical cousin surviving inside this measured stack, not the earlier claim that the hybrid step produced its largest single gain. The memo carries its own counter-datum — NVIDIA's CORTEXA numbers, code-tuned embeddings at 71.95% recall against BM25's 40.67% on SWE-bench file retrieval (secondary, via a practitioner round-up). That does not refute the ablation; the two measure different things. It does show that **BM25's standing is stack-dependent**, which is exactly what a universal claim denies. The node states the contribution where it was measured and claims nothing about stacks nobody measured. **4 · The crossover is stated as dimensions, never as thresholds.** Independent, vendor-neutral head-to-head evaluations at controlled corpus sizes do not exist yet. The crossover the node publishes is a synthesis across unequal sources — it earns the shape of the argument, not a curve. **5 · The RAG analogy is scoped by the fence.** It holds for the verbatim-return portion of a pipeline, which *admits*. Chunk rewriting — contextual retrieval's preprocessing is a model writing new tokens at $1.02 per million document tokens — **mints**, which makes it distillation composed with deferral under ADR 0006. Production stacks are mixed; the classification is per portion, not per product name. ## The two cost shapes The economics the node carries reduce to a shape difference, and both halves are anchored in published prices checked 2026-08-02. **The searched corpus pays per query, in residency.** No build, no hosting, no reindex; the entire bill is search turns admitted to the window at input rates and re-billed on every remaining turn. Order of magnitude: a 5-turn search admitting ~2k tokens per turn, in a session with ~10 turns still to run, is ~10k admitted tokens ≈ 100k token-turns ≈ **$0.10–$0.50 per query** at Haiku 4.5 through Opus 5 input rates ($1–$5 per MTok), before caching. **The indexed corpus pays per corpus, plus a floor.** Embedding is trivia — **$0.02–$0.18 to embed a 1M-token corpus**, from OpenAI's `text-embedding-3-small` at $0.02/MTok to `voyage-code-3` at $0.18/MTok. The bill lives in standing infrastructure: Pinecone serverless Standard runs storage at $0.33/GB/mo and reads at $16–18 per million read units against a **$50/month minimum**, and at skill scale the floor plus the pipeline you now operate — chunking, reindex-on-change, monitoring — is the whole cost. So break-even is query-volume-driven, and at small scale the dollars were never the argument — which is why the one first-party decision on record was made on operational grounds. That is also why caching at 0.1× is load-bearing on the parent's arithmetic rather than a footnote: the competing frontload posture exists only because of it. ## Consequences - No threshold enters `reference-data`. Reinstating one needs a vendor-neutral measurement at controlled corpus sizes, not a restated vendor recommendation. - Only the numbers with published method are quoted in-body: the 5.7%→1.9% ablation, the U-curve, and prices. Everything else travels as direction. - The 2026 agentic-retrieval benchmarks stay cited for direction. A deep read is owed before any of their specific figures enter published prose. - Claims sourced to a party with a commercial stake in the answer are labelled as such wherever they appear, in both directions — the vendor recommending residency and the vendor recommending vectors get the same treatment. ## Related - Reference data — the consuming node; these rulings bind its body. - Deferred context — owns the fetch arithmetic and the caching section the cost shapes above price against. - Cost relocation — owns the admits-vs-mints fence that scopes the RAG analogy. - ADR 0006 — why a chunk-rewriting pipeline is relocation *composed with* transformation rather than a breach of either. - The research memo, 2026-08-02 — the full workshop record: every source, the survey this record compresses, and the questions it left open. [Browse all selected Decisions](https://agentic-atlas.dev/decisions)