You Probably Don't Need a Vector Database Yet

For most RAG systems under roughly 10 to 20 million vectors, pgvector on the Postgres you already run is the correct vector database default in 2026. Skip a dedicated vector database until you can name the specific bottleneck it fixes: filtered-search latency, hybrid search quality, or scale past tens of millions of vectors. Past that line, Qdrant, Pinecone, and Milvus each win for a different, narrow reason. Not because any of them is "better."

I have shipped retrieval systems on both sides of that line: a single Postgres extension handling a few million vectors, and a dedicated cluster handling hundreds of millions. The decision was never about which database scored higher on a benchmark. It was about which specific failure I needed to fix.

Key takeaways

If you read nothing else, read these.

• pgvector is the 2026 default for most RAG systems. If your corpus is under 10 to 20 million vectors, the Postgres you already operate can carry the retrieval layer.
• Vendor benchmarks measure the vendor's data, not yours. A 2025 arXiv analysis found two indexes with identical average recall where one silently returned zero relevant results on nearly 5% of queries.
• Filtered search, not raw recall, is where production breaks. pgvector's own documentation names this: approximate indexes can under-return results once a WHERE clause is applied after the scan.
• Quantization claims and quantization reality are two different numbers. Binary quantization compresses vectors up to 32x at the component level, but real system memory savings, including index overhead, land closer to 7x.
• Migrate on a named bottleneck, not a vibe. "We might scale eventually" is not a bottleneck. "Our filtered queries return 40% fewer results than requested" is.

What a vector database actually does, and when you need one

A vector database stores embeddings, numerical representations of text, images, or other content, and finds the ones closest to a query embedding using approximate nearest neighbor (ANN) search. Instead of scanning every row, it uses an index, usually HNSW, that trades a small amount of accuracy for a large amount of speed. That trade is the entire point: exact nearest-neighbor search does not scale past a few hundred thousand vectors at query latencies anyone will tolerate.

You need a vector database, dedicated or otherwise, the moment you do similarity search at all. You need a dedicated one, separate from your primary datastore, only once your existing database cannot clear a specific, measurable bar. Teams read "vector database" in a tutorial and assume it means new infrastructure, when for most of them it means one extension and one index on a table they already have.

This decision sits downstream of the retrieval pipeline itself. If you have not settled how you chunk documents or when you re-retrieve mid-generation, the database choice is premature. My guide to chunking strategies for RAG and my RAG implementation tutorial both come before this decision, not after it. Get the pipeline right first; the database is one component of it, not the architecture.

The 2026 landscape: pgvector, Pinecone, Qdrant, Weaviate, Milvus

Five names cover almost every production RAG stack I encounter. Each earns its place for a specific reason, not a general one.

• pgvector is a Postgres extension, not a separate system. It supports HNSW indexing, half-precision (halfvec) and binary (bit) vector types, and, since version 0.8.0, iterative index scans that fix the filtered-search under-return problem directly. Its advantage is operational: one database, one backup strategy, joins against your relational data for free.
• Pinecone is fully managed with no infrastructure to operate. You trade ops burden for a monthly bill and less control over index internals. It earns its place when a small team needs scale without hiring for it.
• Qdrant is built for filtered search and quantization from the ground up, with payload indexing designed for the low-selectivity filter problem that breaks naive ANN. It earns its place when your queries are filter-heavy: multi-tenant, permissioned, or date-scoped retrieval.
• Weaviate combines vector and keyword search natively with a modular embedding pipeline. It earns its place when hybrid search quality, not raw speed, is the metric you are optimizing.
• Milvus is built for raw scale: billions of vectors, distributed across nodes, with the operational complexity that implies. It earns its place once you are past what a single well-tuned node of anything else can hold.

Picture a five-person team building a support-ticket RAG assistant. They read a tutorial, reach for Pinecone on day one, and provision an index before writing their first real query. Six months later they have 800,000 vectors. pgvector on their existing Postgres would have carried that load with room to spare, and they would have one less system to pay for, monitor, and back up.

Benchmarks lie: why vendor recall numbers don't transfer to your data

Every vector database vendor publishes a benchmark, run on the vendor's dataset, at the vendor's query distribution. Timescale's own benchmark of pgvectorscale against Qdrant on 50 million 768-dimension Cohere embeddings found 471.57 queries per second at 99% recall, versus 41.47 for Qdrant, an 11.4x gap. Treat that as a vendor claim, not a neutral fact: real and specific, but run on hardware and a query pattern the publisher chose. On the same benchmark, Qdrant held the tail-latency advantage, beating Postgres p99 latency by 48%. Neither number tells you what your own corpus will do.

The sharper problem is not vendor bias. It is what average recall hides. A 2025 arXiv paper, "Towards Robustness: A Critique of Current Vector Database Assessments" (Wang, Zhang, Lu, Chen, Tan), found that ScaNN and DiskANN both score Recall@10 of 0.9 on the MSMARCO benchmark, an identical average. But DiskANN returns zero relevant results on 4.8% of individual queries, a roughly 70x difference in failure rate hidden entirely by the averaged score. The authors propose a new metric, Robustness-δ@K, that measures the fraction of queries falling below a usable recall threshold instead of the mean.

The practical takeaway: compute your own ground truth. Run exact k-nearest-neighbor search on a sample of your real queries against your real embeddings, then measure your candidate index's recall against that, not against a public benchmark. It takes an afternoon and it is the only recall number that means anything for your system.

The decision framework: Postgres-native, dedicated, or fully managed

Vector count is the first filter, but filtering needs and operational appetite decide it alongside scale.

Read the "filtering needs" column as the real decision driver, not the vector count. A 3-million-vector corpus with restrictive tenant and permission filters can break before a 30-million-vector corpus with no filters at all ever does. Count vectors last, not first.

If you are past choosing between two open-source options and into deciding whether to build this internally at all, that build-versus-buy call is exactly what ViitorCloud's custom AI solutions team helps clients make: naming the actual bottleneck before committing engineering months to a migration a config change might have fixed.

When pgvector stops being enough

Three specific bottlenecks justify a dedicated vector database. Anything short of these is premature migration.

Filtered-search selectivity. pgvector's own documentation names the mechanism directly: "with approximate indexes, queries with filtering can return less results since filtering is applied after the index is scanned." Version 0.8.0 added iterative index scans, in strict or relaxed ordering modes, that automatically pull more candidates until the requested count is satisfied. That fix closes most of the gap. It does not fully close it once your filters get restrictive enough, say, a permission check that only 2% of rows satisfy, that the index has to walk a large fraction of the graph to fill the result set anyway.

Hybrid search quality. Combining vector similarity with keyword filtering natively, not as two separate queries merged in application code, is where Weaviate and Qdrant have invested years of engineering. pgvector can approximate this with Postgres full-text search alongside a vector query, but the ranking fusion is your problem, not the database's.

Raw scale past tens of millions of vectors. Single-node Postgres, even well-tuned, has a ceiling. Past it, either index build time becomes unworkable or memory footprint outgrows one machine. That is where Milvus's distributed architecture or Pinecone's managed scaling earns its operational cost.

Memory and cost at scale: what quantization actually costs you

Quantization is the standard answer to vector databases getting expensive at scale, and the headline numbers are real but incomplete. Qdrant's own published example: binary quantization compresses each vector component from 32 bits down to 1 bit, a 32x reduction at that level. Applied to 100,000 OpenAI embeddings at 1,536 dimensions, raw vector storage drops from roughly 900MB to about 128MB. Read that as a 7x overall reduction once HNSW index overhead is counted in, not the 32x the component-level math suggests.

Recall recovery depends on rescoring: comparing the compressed candidates against their original full-precision vectors before returning results. Qdrant's own benchmark hit 100% recall with rescoring enabled at modest oversampling. Skip rescoring and recall degrades, sometimes badly. Binary quantization also has a documented floor: it performs poorly under 1,024 dimensions, since binarizing a short vector discards too much of the signal to recover.

The business consequence is direct. A team choosing binary quantization to hit a memory budget needs to price in the rescoring pass, the extra latency and compute, not just the storage line item. Cut the storage bill by 7x, and query cost can climb back if rescoring runs on every request. Measure the full round trip before committing a quantization scheme to a cost model.

The failure mode: recall that collapses quietly in production

Here is the story I have watched play out at more than one team, in slightly different shapes. The demo retrieves perfectly. Every query in the sample set returns the right document, recall looks great, and the team ships. Then real traffic arrives with real filters: tenant ID, date range, permission scope, exactly the low-selectivity filters that break approximate indexes worst. Recall does not degrade gradually. A specific slice of queries, often the filtered ones, quietly returns garbage, while the aggregate metric barely moves because those queries are a minority of total traffic.

This is the same masking effect the arXiv robustness paper measured directly: an average recall score that looks fine while a meaningful fraction of individual queries fail completely. The fix is not switching databases. It is measuring recall per query segment, not just in aggregate, and specifically measuring the filtered-query segment separately from the unfiltered one.

This failure mode is exactly why I keep coming back to what embeddings can and cannot promise you in the first place. My book Embeddings, Honestly is the honest accounting of what an embedding actually guarantees about similarity, and where that guarantee quietly stops holding once your corpus and your query patterns are not the ones the embedding model was tuned against. The database is downstream of that problem. It cannot fix an embedding that was never going to represent your queries accurately in the first place.

The active-retrieval pattern I wrote about separately makes a related point from the generation side: a system that checks its own confidence and re-retrieves mid-answer catches some of what a single retrieval pass misses. It is a mitigation, not a substitute, for measuring recall honestly on the retrieval layer itself.

Is pgvector good enough for production RAG, or do I need a dedicated vector database?

pgvector is good enough for the large majority of production RAG systems under roughly 10 to 20 million vectors, provided you enable iterative index scans for filtered queries and tune your HNSW parameters against your own recall measurements. Move to a dedicated database only once you can name the specific bottleneck, filtered-search latency, hybrid search ranking quality, or scale past tens of millions of vectors, that pgvector cannot clear.

When should I migrate from pgvector to Pinecone, Qdrant, or Milvus?

Migrate on a measured failure, not a guess: filtered queries under-returning even with iterative scans enabled, hybrid search ranking that application-level fusion cannot fix, or a vector count nearing the memory or index-build ceiling of a single Postgres node. "We might need to scale" is not a migration trigger. "Our filtered recall is 0.61 against a 0.9 target" is.

What's the real difference between Pinecone and Qdrant?

Pinecone is fully managed: no servers to run, less control over index internals, priced for teams that want scale without an ops burden. Qdrant is built around filtered search and quantization, with payload indexing for low-selectivity filters, and it can be self-hosted or managed. Choose Pinecone to remove operational overhead; choose Qdrant when your queries are filter-heavy and you want to tune the index directly.

Does vector quantization (binary or scalar) hurt search accuracy?

It can, but rescoring largely offsets the loss. Binary quantization without a rescoring pass measurably degrades recall, especially under 1,024 dimensions, while a rescoring step against full-precision vectors has recovered close to full accuracy in published benchmarks. Scalar quantization is a gentler compromise: typically less recall lost than binary, at a smaller memory saving. Test both against your own ground truth before choosing either.

Choosing a vector database is a bottleneck-naming exercise before it is a technology choice. If you are past the point where a config change fixes it, and want a team that has made this exact call on production systems, ViitorCloud builds and ships custom RAG and retrieval systems, measuring the real bottleneck before it recommends a migration, evals included from the start.