“Which vector database should we use?” is the wrong first question. We hear it at the start of almost every retrieval-augmented generation project, usually before anyone has written down what recall the application actually needs, how often the corpus changes, or who will own the cluster at 3 a.m. six months from now. The comparison tables that circulate online answer a question nobody is really asking — which engine has the most features — and skip the three variables that decide the outcome.
The decision that matters is narrower and more boring: given your corpus size, your write pattern, your filtering requirements, and your appetite for operating a stateful distributed system, which engine’s failure modes can you live with? Pick on that basis and the shortlist collapses to two or three candidates quickly. Pick on feature checklists and you end up migrating in month nine, which is the failure class this article is about.
The failure this decision usually produces
The specific way vector-database selection goes wrong is not “we chose a slow engine.” It is a re-platforming forced by a requirement that was invisible at selection time. Three variants show up repeatedly in our engagements (observed across TechnoLynx retrieval projects; not a benchmarked rate):
The first is metadata filtering. A team builds a semantic search prototype over a flat corpus, everything works, and then the product requires results scoped to a tenant, a date range, and a document-permission set. Pre-filtering with high selectivity breaks approximate-nearest-neighbour graph traversal — an HNSW graph whose eligible nodes are 0.1% of the index no longer has a connected path to the true neighbours, so recall falls off a cliff even though latency looks fine. Engines differ enormously in how they handle this, and it is almost never tested in the prototype.
The second is the write pattern. Vector indexes are cheap to query and expensive to mutate. An IVF-style index built with a fixed set of centroids degrades as the data distribution drifts away from the vectors the centroids were trained on; graph indexes handle incremental inserts better but accumulate deleted-node tombstones that must eventually be compacted. If your corpus turns over weekly, index rebuild cost is a first-order operating expense, not a footnote.
The third is operational ownership. A distributed vector database is a stateful system with its own consensus, sharding, and backup semantics. Teams routinely choose a horizontally scalable engine for a workload that fits in a single node’s memory, then discover they have inherited an operations burden with no corresponding benefit. The reverse — outgrowing an embedded library — is easier to recover from than being under-staffed for a cluster.
Getting this right the first time is worth real money. Migration between vector engines means re-embedding or re-ingesting the corpus, re-tuning index parameters, re-validating recall, and running both systems in parallel during cutover. On a corpus in the tens of millions of vectors, that is typically weeks of engineering time plus duplicated infrastructure spend — the sort of avoidable cost that also shows up in our work on cloud cost reduction strategies.
The four variables that actually determine the choice
Before comparing engines, write down four numbers. If you cannot, you are not ready to choose.
- Corpus size and vector dimensionality. A million 768-dimensional float32 vectors is roughly 3 GB of raw vectors before index overhead; HNSW typically adds a further 30–60% depending on graph degree. Ten million 1536-dimensional vectors is a different architecture, not a bigger version of the same one.
- Write-to-read ratio and freshness requirement. Append-only with hourly batches is a different engine choice than continuous upserts with sub-second visibility.
- Filter selectivity. What fraction of the index survives your typical metadata filter? Above ~10% most engines cope; below ~1% the filtering strategy dominates everything else.
- Recall target. Not “high accuracy” — a number, measured against exact brute-force search on a held-out query set. Most teams discover they need recall@10 of about 0.90–0.95, and that the last few points cost disproportionately in latency and memory.
Comparison matrix: open source vector engines by decision axis
The matrix below reflects architectural properties and published documentation for each project, plus patterns we have seen in deployment. It is deliberately not a benchmark table — throughput numbers without your data distribution, your filter selectivity, and your hardware are marketing, not evidence.
| Engine | Architecture | Where it fits | Filtered-search behaviour | Main operational cost |
|---|---|---|---|---|
| FAISS (library) | In-process index library; IVF, HNSW, PQ variants | Batch retrieval, research, embedded in a service you already operate | You implement filtering yourself, usually post-filter with over-fetch | You own persistence, sharding, and updates entirely |
| pgvector (PostgreSQL extension) | HNSW/IVFFlat inside Postgres | Corpora up to low tens of millions where relational data and vectors co-exist | Strong — the planner combines SQL predicates with vector search; genuinely good at high-selectivity filters | Postgres tuning (maintenance work-mem, autovacuum) becomes vector-index tuning |
| Qdrant | Purpose-built, Rust; HNSW with payload indexes | Filter-heavy production search, multi-tenant apps | Designed for it — payload indexes and filter-aware graph traversal | A service to run, but a comparatively simple one |
| Milvus | Distributed, separated compute/storage, multiple index types | Hundreds of millions to billions of vectors, GPU-accelerated indexing | Supported via scalar fields and partitions; partition design matters a lot | Highest — several components, object storage, message queue |
| Weaviate | Purpose-built with schema, modules, hybrid search | Teams wanting built-in hybrid keyword+vector and a schema model | Inverted index plus HNSW; hybrid ranking is native | Moderate; module/schema coupling adds migration friction |
| Elasticsearch / OpenSearch | Lucene HNSW alongside BM25 | Organisations already running the cluster for keyword search | Excellent filter integration; hybrid search is the native case | You already pay it — that is the argument |
| Chroma / LanceDB | Embedded / file-oriented | Prototypes, single-node apps, notebook-to-service paths | Basic to moderate | Almost none, until you outgrow the node |
The last row deserves emphasis. If your organisation already operates an Elasticsearch or OpenSearch cluster with real expertise attached, dense-vector fields in Lucene are frequently the correct answer even when a purpose-built engine would score better in isolation — the marginal operational cost is close to zero. We wrote separately about where Elasticsearch and Solr fit among NoSQL stores for big-data workloads; that article covers the general search-and-analytics case, while this one is specifically about the vector-retrieval decision.
How should you test a vector database before committing?
Run a bounded evaluation, not a bake-off. The following sequence takes a couple of days per candidate and answers the questions that matter.
- Build a ground-truth set. Take 500–1,000 real queries, compute exact nearest neighbours by brute force over the full corpus (FAISS
IndexFlatIPis fine for this), and store them. Without this you cannot measure recall, and without recall your latency numbers mean nothing. - Measure recall and latency together, with filters on. Report recall@k at a fixed p95 latency budget, at your real filter selectivity. A candidate that hits 0.97 recall unfiltered and 0.6 recall at 1% selectivity has failed.
- Test the write path at production ratio. Ingest while querying. Watch what happens to recall and p99 latency during a bulk upsert and during a large delete followed by compaction.
- Time a full index rebuild. Then decide whether you can afford it monthly. This number drives more architecture decisions than anything else in the list.
- Kill a node. Or restart the container. Confirm what the recovery time and data-loss window actually are, rather than what the documentation implies.
- Price the steady state. Memory-resident index size × replicas, plus rebuild compute. Compare against the same corpus in the system you already run.
If a candidate survives all six, the remaining differences are usually ergonomic — client libraries, schema handling, how much the API fights your framework — and ergonomics is a legitimate tiebreaker once correctness and cost are settled.
Worked example: 20 million chunks, multi-tenant, weekly refresh
Assume a document-retrieval product: 20 million text chunks, 1024-dimensional embeddings, tenant-scoped queries where a typical tenant holds 0.5% of the corpus, 60 queries per second at peak, recall@10 target of 0.93, and a weekly full re-embedding because the embedding model is still changing.
Raw vectors at float32 are roughly 80 GB. That immediately rules out comfortable single-node in-memory HNSW on commodity hardware unless you quantize; scalar quantization to int8 brings the vector payload to roughly a quarter of that, at a recall cost that has to be measured rather than assumed. The 0.5% filter selectivity is the decisive constraint: it eliminates any engine relying on post-filtering with over-fetch, because retrieving enough candidates to find ten tenant-scoped neighbours would mean scanning a large fraction of the index.
That leaves engines with filter-aware traversal or partition-per-tenant designs. Qdrant with tenant as an indexed payload field and Milvus with tenant partitions are both defensible; pgvector becomes plausible if tenants can be physically partitioned into separate tables. The weekly rebuild then decides it — an engine with GPU-accelerated index building and a clean blue/green collection swap makes a weekly 20-million-vector rebuild routine, whereas one requiring in-place reindexing under live traffic does not. Note what did the work here: four numbers, not a feature list.
What the comparison tables systematically omit
Three things.
Embedding model coupling. Your retrieval quality is dominated by the embedding model, not the index. A better model with a mediocre index beats a mediocre model with a perfectly tuned index almost every time. Changing embedding models means re-embedding everything, so the ability to hold two collections side by side and cut over cleanly is worth more than a 15% latency edge.
Hybrid retrieval. Pure dense retrieval underperforms on exact-match queries — product codes, names, acronyms. In practice most production systems that started dense-only end up adding BM25 and reciprocal-rank fusion. If you know you will need hybrid search, engines where it is native rather than bolted on save a rewrite.
The licence and hosting question. “Open source” spans permissive Apache-2.0 projects, source-available licences with commercial restrictions, and open cores whose interesting features live in the managed tier. Deploying the open engine and later needing the managed features is a variant of the lock-in problem we discuss in the context of SaaS, DaaS and private-SaaS migration trade-offs. Read the licence before the benchmark.
FAQ
Which open source vector database is best for AI workloads?
There is no single best engine — the choice is determined by corpus size, filter selectivity, write pattern, and how much operational capacity you have. For filter-heavy multi-tenant search, filter-aware engines such as Qdrant or partitioned Milvus fit; for corpora in the low tens of millions alongside relational data, pgvector inside PostgreSQL is often sufficient; if you already operate Elasticsearch or OpenSearch with real expertise, Lucene dense-vector fields are frequently the rational choice because the marginal operational cost is near zero.
Do I need a dedicated vector database, or is a Postgres extension enough?
pgvector handles corpora up to the low tens of millions of vectors well, and it wins decisively when your vectors live next to relational data you already query, because the planner can combine SQL predicates with vector search. You outgrow it when index memory exceeds what one instance can hold comfortably, when rebuild time under live traffic becomes unacceptable, or when you need GPU-accelerated index construction.
How does metadata filtering break approximate nearest neighbour search?
Graph-based indexes such as HNSW find neighbours by traversing edges between vectors. When a filter removes most of the index, the eligible nodes may no longer form a connected path, so traversal terminates early and returns poor results — latency stays low while recall collapses. Engines that index filter payloads and make traversal filter-aware avoid this; engines relying on post-filtering with over-fetch degrade sharply below roughly 1% selectivity.
What should I measure before choosing a vector database?
Build a ground-truth set of 500–1,000 real queries with exact nearest neighbours computed by brute force, then measure recall@k at a fixed p95 latency budget with your real filters applied. Add three operational measurements: recall and p99 latency during concurrent ingest, wall-clock time for a full index rebuild, and recovery behaviour after killing a node. Those five numbers separate candidates; feature matrices do not.
How much does migrating between vector databases actually cost?
Migration means re-ingesting or re-embedding the corpus, re-tuning index parameters, re-validating recall against ground truth, and running both systems in parallel during cutover. On a corpus of tens of millions of vectors that is typically weeks of engineering effort plus duplicated infrastructure spend — which is why the selection criteria above are worth applying before the prototype hardens into production.
The number that should decide it
If we had to reduce this to one instruction: measure recall at your real filter selectivity, on your real corpus, before you read another comparison table. Almost every vector-database regret we have seen traces back to a prototype that was evaluated unfiltered on a subset, and a production system that then had to answer scoped queries over the whole thing. The engines differ, sometimes a lot — but the variable that decides which difference matters is one you control and can measure this week. Our work on retrieval and cloud data architecture starts from that measurement rather than from the shortlist.
