The slide in the Tuesday architecture review reads: "Vector DB — Pinecone (TBD)." That parenthetical TBD is doing more work than anyone in the room understands. By the time the agent ships, the choice will determine the recurring infrastructure bill, the on-call rotation, whether your in-house data team can pull the index for an audit, and whether you can sleep through a 3 a.m. PagerDuty alert. The wrong choice for a 100,000-document corpus costs $14,000-$48,000 per year you did not need to spend, plus an engineer-quarter of bolted-on backfill code. The right choice for the same workload costs $80-$400 per month, ships in an afternoon, and has zero ops burden. This lesson is the decision matrix that gets you to that right choice without becoming an infrastructure engineer.

Why the Default Pinecone Pick Is Often Wrong

Pinecone is the brand name in vector databases the way Stripe is the brand name in payments — most operators reach for it because it is what they have heard of. That instinct made sense in 2022 when the market had three serious vendors and Postgres had no usable vector extension. In 2026 it is, more often than not, the most expensive way to solve a problem that has cheaper, equally-capable answers.

A 100,000-document corpus is not a "vector database scale" problem. After chunking to roughly 500-token segments, you end up with somewhere between 600,000 and 1.2 million vector embeddings depending on chunk overlap. At a 1024-dimension embedding (Cohere's embed-v4 default, Voyage's voyage-3, OpenAI's text-embedding-3-large at the small variant) you are storing 4-5 GB of vector data plus metadata. That fits on a single Postgres instance, on a Turbopuffer namespace at twenty dollars a month, on a Chroma SQLite file your laptop can serve. The problem is small. The market sells you a big-problem product because it is what they have built.

The first question is never "which vector database is best." It is "how big is my retrieval problem in 2026, and does it actually need a vector database service?" For most operators with under 5 million chunks, the answer is no.

The 2026 vendor landscape, briefly

Six categories of options compete for the 100K-document workload. They are not interchangeable, and the difference matters when you write the check.

• Managed vector services: Pinecone, Weaviate Cloud, Qdrant Cloud. Hosted, multi-tenant, SLA-backed. You ship vectors, they ship results.
• Postgres + extension: pgvector (the de-facto extension), pgvectorscale for higher recall, Supabase as a managed wrapper. Same Postgres you already operate.
• Object-store-native: Turbopuffer (the standout in 2025-2026), LanceDB on S3. Vectors live in object storage with a thin index in memory.
• Embedded/in-process: ChromaDB, LanceDB local, FAISS-on-disk. The database runs inside your worker process. Zero network hop.
• Platform-native: Relevance AI's built-in store, Vellum's Document Index, Glean's enterprise search, Sana AI's knowledge graph. Vector storage is a feature inside a larger agent platform.
• Hyperscaler offerings: AWS OpenSearch Serverless (vector mode), Azure AI Search, Google Cloud Vertex AI Vector Search. The "we already pay them" option.

Each one has a use-case in which it is the cheapest, fastest, lowest-ops choice. The rest of the lesson is when each is that thing and when it is a trap.

The Six Decision Axes That Matter

Pick a vector store on these six axes, in this priority order. Anyone who tells you the "best" vector store without knowing your answers to these is selling you something.

Axis 1: How many vectors will live in the index in 18 months

The 18-month horizon matters because retrieving from production is a one-way door — once you have written 5 million chunks, the migration cost to a different store is 2-6 engineer-weeks. The corpus doubles in size in the year after launch because (a) your team adds new sources you forgot existed, (b) the chunker gets tuned to produce 30-50% more chunks per document, and (c) feedback loops add re-embedded variants.

Volume ranges and what they actually mean in 2026:

• Under 500K vectors: Anything works. Pick the cheapest, fastest-to-ship option. ChromaDB on a single VM, pgvector, Turbopuffer free tier. Do not even think about managed services.
• 500K-5M vectors: The most common "real" operator workload. pgvector with the right index parameters, Turbopuffer, or Pinecone Serverless. Hyperscaler vector search becomes viable.
• 5M-50M vectors: Managed services start to pay off. Pinecone, Qdrant Cloud, Turbopuffer at scale. pgvector still works with careful tuning and partitioning, but you are inviting an on-call rotation.
• Over 50M vectors: You have an engineering problem, not an operator problem. Hire a vector-search engineer or use a hyperscaler with a dedicated cluster. This lesson is not aimed at you.

A 100K-document corpus chunked at 512 tokens with 15% overlap yields roughly 800K-1.2M vectors. That is the second bucket. The cheap-and-easy options are exactly the ones that fit.

Axis 2: Read latency budget, p95

The retrieval call is one step in a multi-step agent loop. Whatever you spend on retrieval, you do not spend on LLM inference. The total user-facing latency budget for a single-turn agent response in 2026 is roughly 2-4 seconds at the p95. The LLM dominates: 800-2500 ms for a Claude Sonnet or GPT-5 call with reasonable output length. Retrieval has to fit in a 200-400 ms p95 window, including reranking.

Vendor-published p95 latencies for a 1M-vector index with top-10 retrieval, May 2026:

• Pinecone Serverless: 40-90 ms p95 (within-region)
• Qdrant Cloud: 30-70 ms p95
• Turbopuffer: 80-150 ms p95 (object-store cold reads can spike to 300 ms)
• pgvector with HNSW index: 25-80 ms p95 on a properly-sized instance
• Weaviate Cloud: 50-120 ms p95
• ChromaDB (local): 5-30 ms p95 (no network hop)
• OpenSearch Serverless: 100-250 ms p95 (the slowest of the hyperscalers)

The number that surprises operators: cold-start latency on object-store-backed stores. Turbopuffer is wonderful for cost but the first query against a namespace that has been idle for ten minutes can spike to 300-500 ms while indexes warm. For a conversational agent that is fine. For a real-time customer-facing flow it is a problem you have to design around with keep-warm pings or a hybrid hot-tier.

Axis 3: Write/update frequency

How often does the corpus change? Three regimes, very different vendor implications.

• Bulk reindex (weekly or less): All vector stores handle this. Turbopuffer and object-store-native options shine here because you upload a batch and pay almost nothing for storage.
• Streaming updates (hourly to daily): Most operator workloads. pgvector, Pinecone, Qdrant all handle this. Watch for index rebuild costs on HNSW-based stores — pgvector with HNSW does incremental updates correctly but the index quality degrades over time without periodic rebuilds.
• Near-real-time (sub-minute): Customer support agents where a new ticket needs to be retrievable within 60 seconds. Pinecone, Qdrant, and Vespa hold up. pgvector struggles past 50 writes/second without specialized tuning. Turbopuffer is not designed for this and the docs say so.

Axis 4: Metadata filtering complexity

The retrieval call almost never says "find me the top-10 most similar vectors." It says "find me the top-10 most similar vectors where tenant_id = 'acme-corp' AND document_type IN ('contract', 'sow') AND last_modified > 2025-01-01." That is the metadata filter, and the way each vendor handles it varies enormously.

• pgvector: Filters are just SQL WHERE clauses. The optimizer picks an index. Highly expressive but you are responsible for index design.
• Pinecone: Metadata filters are a first-class feature with a JSON-like query language. Strong on equality and IN clauses, weaker on range queries and full-text. No JOIN — you cannot filter by another table.
• Qdrant: Among the strongest filter engines, includes geo, datetime, and nested field matching.
• Weaviate: GraphQL-style filter language. Powerful but adds a learning curve.
• Turbopuffer: Excellent filtering, designed around it. Filters are partitioned to keep cost low even on huge filter sets.
• ChromaDB: Basic filtering. Equality and IN. Anything more complex you do post-fetch.

If your retrieval has multi-tenant isolation requirements, complex date ranges, or hybrid keyword-plus-vector search, pgvector and Qdrant are usually the right answer. If filtering is mostly tenant ID and a tag list, every option works.

Axis 5: Ops burden — who runs the database

This is the axis operators systematically under-weight. The choice is who carries the pager.

• You/your team carries the pager: Self-hosted Qdrant, self-hosted Weaviate, pgvector on your own Postgres, FAISS on your own VM. Cheap. Operational burden: high. Backups, replication, version upgrades, index corruption, capacity planning.
• Managed service carries the pager: Pinecone, Qdrant Cloud, Weaviate Cloud, Turbopuffer. You pay them, they handle uptime. SLA is 99.9% or better.
• Hyperscaler carries the pager: AWS OpenSearch Serverless, Azure AI Search, Google Vertex AI Vector Search. Inside the cloud you already have. Auth and networking integrated.
• Platform carries the whole stack: Relevance AI, Vellum, Glean. You upload documents; they handle chunking, embedding, retrieval, citations. No pager at all. You give up control of retrieval logic.

For most operator teams the right answer is "managed service" or "platform-native." The total cost of an on-call rotation for a single self-hosted vector database is 1-2 engineer-weeks per quarter on average. That is twenty to forty grand of fully-loaded engineering cost annually, which dwarfs the difference between any two managed offerings.

Axis 6: Where the data lives (compliance and sovereignty)

If your corpus contains EU personal data and the agent runs from a US-region service, you have an Article 44 GDPR problem before you have a retrieval problem. Vendors and their region availability:

• Pinecone: US, EU (Frankfurt), AP regions. SOC 2 Type II, HIPAA, GDPR DPA available.
• Weaviate Cloud: AWS regions in US, EU, AP. HIPAA-capable on Enterprise tier.
• Qdrant Cloud: AWS, GCP, Azure regions across US, EU, AP. Self-host option for full sovereignty.
• Turbopuffer: S3 + AWS regions. EU regions added in 2025.
• pgvector on Supabase: Region-pinned at project create time. Good GDPR story.
• Relevance AI / Vellum: US-only by default in May 2026. Check before signing.
• Hyperscaler offerings: Wherever your existing tenancy lives.

The DPO (Data Protection Officer) will ask this question. Have the answer ready before you onboard the vendor.

The 100K-Document Decision Matrix

For the specific workload framed at the top — 100,000 documents, ~1M chunks, mixed update cadence, customer-facing agent — here is the matrix that compresses six axes into a recommendation.

Cost comparison, May 2026 pricing, 1M vectors at 1024 dimensions

All numbers are list prices for a workload of 1 million vectors, 100K queries per month, moderate metadata filtering, single region. Negotiated enterprise pricing varies.

• Pinecone Serverless: $80-$140/month for read+write at this volume. Predictable. Scale linearly.
• Pinecone Pod-based (legacy): $350-$700/month for an s1 pod. Avoid unless you have a specific reason.
• Qdrant Cloud (managed): $90-$170/month for the equivalent cluster.
• Weaviate Cloud (Sandbox tier): Free up to 100K vectors. Standard tier $25-$60/month for 1M vectors.
• Turbopuffer: $20-$60/month for 1M vectors at this query volume. Object-store pricing is genuinely disruptive.
• pgvector on Supabase Pro: $25/month base + $0.125/GB storage. Roughly $40-$80/month for this workload. Caveat: you share the Postgres instance with the rest of your app.
• pgvector on dedicated AWS RDS db.t3.large: $135/month + storage. Comfortable. Reserved instances drop this to $80.
• ChromaDB self-hosted on a $20 VM: $20/month. Plus the engineer-time to manage it.
• AWS OpenSearch Serverless: $250-$700/month minimum given the OCU pricing model. Cost-effective only if you already use it for other workloads.
• Relevance AI / Vellum: Pricing rolled into the platform subscription. Effectively $200-$1,500/month depending on the tier.

The matrix at a glance

For a 100K-document, customer-facing operator agent in 2026, the defensible default order is:

1. If your team already runs Postgres: pgvector on Supabase or your existing RDS. Lowest friction. Same auth, same backups, same SQL. Adds twenty minutes of setup time. Hits all six axes adequately.
2. If you want the lowest dollar cost and can tolerate 100-200 ms p95 retrieval: Turbopuffer. Cheapest by a meaningful margin. Surprisingly good filter support. Watch the cold-start.
3. If you need rock-solid sub-100 ms latency and are spending less than $300/month anyway: Pinecone Serverless or Qdrant Cloud. Pinecone for brand-safety in the architecture review; Qdrant for slightly better filter expressiveness.
4. If you are building inside Relevance AI, Vellum, or a similar platform: Use the platform-native store. Switching costs are real and the platform is bundling for a reason.
5. Avoid by default: AWS OpenSearch Serverless (overpriced for this scale), self-hosted Weaviate (operationally heavy), and Pod-based Pinecone (the old pricing model).

The Real Cost of Each Pick (Not the List Price)

List price is the start of the cost conversation. There are three other cost categories operators consistently miss.

Embedding cost dwarfs storage cost

You pay the embedding provider per token of input. For a 100K-document corpus, full reindex with Cohere embed-v4 (May 2026 pricing: $0.12 per 1M input tokens for the small tier, $0.18 for large) costs $40-$90 for the first index build. With OpenAI text-embedding-3-large at $0.13 per 1M tokens, it is in the same range. With Voyage voyage-3 at $0.10 per 1M, slightly less.

That is one-time. The recurring cost depends on how often the corpus changes. If 5% of documents get re-embedded per week, your annual embedding bill is $100-$250. Trivial compared to most vendor pricing.

The hidden trap: if you switch embedding models (you will, at least once in the agent's lifetime, as new models ship) you re-embed the entire corpus. Budget this as a quarterly expense even if you do not plan to switch. Anthropic, Cohere, OpenAI, and Voyage all shipped at least one new embedding model in the last twelve months.

Re-ranker cost is its own line item

The re-ranker step (covered in lesson 3 of this chapter) calls Cohere Rerank 3 or Voyage Rerank-2 on every query. At Cohere's $2.00 per 1K reranks pricing for May 2026, a 100K queries-per-month workload costs $200/month for reranking alone. This is often more than the vector store itself. Budget the reranker as a peer cost of the vector DB.

Egress and cross-region cost

If your vector DB lives in a different cloud region than your agent compute, you pay egress on every query response. At AWS's $0.02/GB cross-region pricing and a 50-vector response per query, 100K queries pull about 2 GB per month — $40 in egress alone. Not a deal-breaker, but a line item people miss. Always co-locate the vector store with the agent compute.

The Three Traps the Default Pick Falls Into

Trap one: optimizing for vector-database benchmarks not retrieval quality

Vendor benchmark pages compete on raw retrieval latency at top-10. Your agent quality competes on whether the retrieved chunks are actually relevant to the user query. A 30 ms retrieval that returns the wrong chunks is worse than a 150 ms retrieval that returns the right ones. Pick the vendor on metadata, filter expressiveness, hybrid (keyword + vector) search support, and recall — not raw latency.

Specifically: hybrid search (BM25 plus vector) consistently outperforms vector-only retrieval on real corpora by 10-25% on hit-rate-at-k. Weaviate, Qdrant, and pgvector (with the pg_trgm extension or full-text search) support this natively. Pinecone added hybrid in 2024 but the implementation is less mature. ChromaDB requires you to bolt it on. Make hybrid support a requirement, not a nice-to-have.

Trap two: the lock-in by SDK

Pinecone's SDK, Weaviate's GraphQL client, Qdrant's REST API — each is different. If you write your retrieval layer against one vendor's SDK, switching costs go up. The defense is a thin abstraction layer: a single retrieve(query, filters, k) -> chunks function in your codebase that hides the vendor. Eight lines of code on day one save you two engineer-weeks on the day you switch. The LangChain, LlamaIndex, and Haystack abstractions provide this for free if you tolerate the framework dependency.

Trap three: ignoring the platform-native option

If your agent runs inside Relevance AI, Vellum, Glean, or Sana AI, those platforms ship with a vector store. The vendor has already made every decision in this lesson for you. The defaults are reasonable for the corpus sizes their customers ship. Adopting an external vector DB inside one of these platforms means writing custom connectors, managing two systems, and giving up the platform's citation and observability features. For 80% of operator workloads on those platforms, the platform-native store is the right answer. Reach for the external DB only when you have a specific requirement the platform cannot meet — multi-region replication, very large scale, specific compliance certifications.

The Decision Template (Fifteen Minutes, Not Fifteen Meetings)

Run this template, answer it once, attach the document to the architecture review. The decision is then made and defensible.

1. Corpus size projection. Document count today and in 18 months. Multiply documents × chunks-per-document (typically 8-12) for a vector count estimate.
2. Latency budget. What is the p95 retrieval latency you can spend, given your total user-facing latency budget? Subtract LLM call (1500 ms typical) and reranker (50-100 ms) from your total.
3. Write cadence. Bulk weekly, streaming hourly/daily, or near-real-time sub-minute? Pick one.
4. Filter requirements. List the metadata filters your retrieval queries use. Multi-tenant ID? Date ranges? Hybrid keyword + vector? Boolean logic?
5. Ops budget. Engineer-weeks per quarter you can spend on the vector store. Honest answer, not aspirational.
6. Compliance region. EU? US? Both? Specific cert requirements (HIPAA, SOC 2, ISO 27001)?
7. Existing infra. Do you already run Postgres? AWS? GCP? Inside a Relevance AI / Vellum tenancy?

With those seven answers, the right vector store falls out in fifteen minutes. The template is also the document you hand to your security review team to head off the inevitable "why this vendor" question six months later.

Key Takeaways

• The default Pinecone pick is often wrong in 2026 for a 100K-document corpus. The workload (roughly 1M vectors) fits comfortably on pgvector, Turbopuffer, or platform-native stores at one-third to one-tenth the recurring cost.
• Decide on six axes in priority order: vector count at 18 months, p95 latency budget, write cadence, metadata filter complexity, ops burden (who carries the pager), and data sovereignty region.
• For 100K documents (~1M vectors) the defensible defaults are: pgvector on Supabase if you already run Postgres; Turbopuffer if you want lowest cost and can tolerate 100-200 ms p95; Pinecone Serverless or Qdrant Cloud if you need rock-solid sub-100 ms; platform-native if you build inside Relevance AI / Vellum / Glean.
• Avoid AWS OpenSearch Serverless at this scale (overpriced), self-hosted Weaviate (heavy ops), and Pinecone Pod-based pricing (legacy model).
• List price is one cost; embedding (re-embedding when models change), reranker calls ($200/month at 100K queries with Cohere Rerank 3), and cross-region egress add their own line items. Budget all four.
• Hybrid search (BM25 + vector) outperforms vector-only by 10-25% on hit-rate-at-k on real corpora. Make hybrid a requirement, not a nice-to-have. Weaviate, Qdrant, and pgvector support it natively.
• Defend against vendor lock-in with a thin retrieve(query, filters, k) abstraction. Eight lines of code on day one save two engineer-weeks on the day you switch vendors.
• If you build inside a platform (Relevance AI, Vellum, Glean, Sana AI), use the platform-native store. The platform has bundled retrieval, citations, and observability; replacing the store gives up that integration.
• The ops burden of a self-hosted vector DB is 1-2 engineer-weeks per quarter. That fully-loaded cost typically exceeds the price difference between managed offerings. Pay for managed unless you have a specific reason not to.
• Run the seven-question decision template once, attach it to the architecture review, and the choice is made and defensible. The same document answers the security review six months later.