“We’re picking a NoSQL database, and we’ve shortlisted MongoDB, Cassandra, and Elasticsearch.”

That shortlist is where a lot of big-data architectures go wrong — not because Elasticsearch is a bad choice, but because it is not the same kind of thing as the other two. Elasticsearch and Apache Solr are search engines that happen to store documents. MongoDB and Cassandra are databases that happen to support queries. The word “NoSQL” covers both, which is exactly why it is a poor basis for a decision.

The practical consequence shows up six months later, when someone asks whether the search index can be treated as the authoritative copy of the data. If the architecture was chosen on the strength of the NoSQL label alone, the answer is usually “we assumed so”, and that assumption is the failure.

What “NoSQL” actually stopped meaning

The term was useful in about 2009, when it distinguished “not a relational database with SQL and joins” from everything else. It has since fragmented into at least five families with almost nothing in common beyond the negation:

• Key-value stores — Redis, DynamoDB. Constant-time access by key, no query planner worth the name.
• Wide-column stores — Cassandra, HBase, ScyllaDB. Partition-key-driven, tuned for write throughput at scale.
• Document stores — MongoDB, Couchbase. Flexible schemas, secondary indexes, transactional guarantees that have improved substantially since the early releases.
• Graph databases — Neo4j, JanusGraph. Traversal-first; the query pattern is the data model.
• Search engines — Elasticsearch, OpenSearch, Apache Solr. Inverted-index-first, built for relevance ranking and full-text retrieval over text and structured facets.

Only the last family is optimised for the question “which documents are most relevant to this messy human query?” All the others are optimised for “give me this record, or these records, cheaply and reliably.”

Grouping them under one shortlist implies they are substitutes. They are layers. In most big-data systems we work on, two or three of these families coexist, and the interesting design work is in the boundaries between them — not in picking a winner.

Elasticsearch and Solr: both Lucene, different centres of gravity

Both Elasticsearch and Solr sit on top of Apache Lucene, the Java library that implements the inverted index, scoring (BM25 by default in modern versions), and the segment-merge machinery underneath. That shared foundation means their core retrieval behaviour is more similar than most comparison posts admit. Tokenisation, analysers, stemming, BM25 relevance — largely the same engine.

The divergence is in operations, governance, and ecosystem.

Solr came out of the Apache ecosystem and behaves like it: configuration is explicit, schema definition is a first-class artifact, and SolrCloud coordinates through Apache ZooKeeper. Teams that want the index definition under version review, reproducible from a config directory, tend to find Solr’s posture comfortable.

Elasticsearch grew up as an API-first product with a much larger commercial surface — the ingest pipelines, Kibana, alerting, the observability stack. Its licence changed in 2021 from Apache 2.0 to the Elastic Licence (SSPL/ELv2 dual), which is what triggered AWS’s OpenSearch fork. Anyone evaluating Elasticsearch in 2026 is really evaluating three options: Elastic’s distribution, OpenSearch, or Solr. That is a licensing and vendor-dependency question at least as much as a technical one, and it belongs in the same conversation as the broader SaaS, DaaS and private-SaaS lock-in trade-offs any cloud data platform faces.

Decision table: which store owns which job

The table below is the shortlist we would actually put in front of an architecture review. Evidence class for the “typical fit” column is observed-pattern — drawn from platforms we have built and inherited, not from a published benchmark suite.

Job to be done	Right family	Reasonable picks	Wrong pick, and why
System of record for transactional entities	Relational or document store	PostgreSQL, MongoDB	Search engine — no durable transaction boundary you’d want to bet money on
Full-text relevance ranking over documents	Search engine	Elasticsearch, OpenSearch, Solr	Document store — regex/$text queries degrade badly past a few million docs
Faceted navigation on a catalogue	Search engine	Solr, Elasticsearch	Wide-column store — no aggregation model for facet counts
Very high-volume time-ordered writes	Wide-column / time-series	Cassandra, ClickHouse, TimescaleDB	Search engine — indexing cost per write is roughly an order of magnitude higher (observed-pattern)
Log and metric search across a fleet	Search engine	Elasticsearch/OpenSearch + Kibana	Relational — full-text scan cost is prohibitive
Semantic / embedding retrieval for RAG	Vector index (standalone or embedded)	pgvector, Qdrant, Elasticsearch kNN	Pure keyword index — lexical match misses paraphrase
Sub-millisecond point lookups by key	Key-value	Redis, DynamoDB	Search engine — query parse plus coordination adds fixed overhead
Relationship traversal (fraud rings, supply graphs)	Graph	Neo4j	Document store — multi-hop joins in application code

Read the table as a routing device. The recurring failure we see is one row being answered by a store chosen for a different row.

Why is Elasticsearch a bad system of record?

This is the question worth asking out loud, because the answer is not “it loses data”. Modern Elasticsearch and OpenSearch are considerably more durable than their 2015 reputations suggest — translog, replica shards, sequence numbers and primary terms all exist and work.

The problem is what durability is for. A search index is a derived, lossy, re-buildable projection of the truth:

• Analysers destroy information deliberately. Lowercasing, stemming, stop-word removal — the indexed form is not the input form. You store the original in _source, but _source is a convenience field, not a storage engine.
• Mapping changes usually mean reindexing. Change an analyser or a field type and you rebuild. If the index is the only copy, “rebuild” has nothing to rebuild from.
• No cross-document transactions. Refresh intervals mean a write is visible when the segment refreshes, typically a second by default. That is fine for search and unacceptable for a balance transfer.
• Relevance is a moving target. You will tune scoring, add synonyms, change field boosts. Every one of those is a schema-affecting change to something you also depend on for correctness.

The correct frame: a search index should always be reconstructible from a source of truth you control. If losing the cluster means losing the data, the architecture has one fewer layer than it needs. That reconstructibility requirement is also what makes search infrastructure cheap to run aggressively — you can size for performance rather than paranoia, which connects directly to the cloud cost-reduction strategies that treat stateless-and-rebuildable tiers differently from stateful ones.

The indexing pipeline is the real design problem

Once you accept the index as a projection, the interesting engineering moves upstream. How does data get from the system of record into the index, and how do you know it arrived?

Three patterns dominate, in rough order of how much operational maturity they require:

1. Dual write from the application. Simple, and quietly wrong. The two writes are not atomic; any failure between them leaves the index stale with no signal.
2. Change data capture. Debezium reading the database write-ahead log into Kafka, consumers projecting into the index. Ordering and replay come free, which matters enormously when you need to rebuild.
3. Scheduled batch reindex. A Spark or Flink job rebuilding from the warehouse. Coarse, slow, and the most reliable of the three for correctness.

Most production systems we have worked on end up running (2) for freshness and (3) as the periodic reconciliation that catches whatever (2) missed. The reconciliation job is the part teams skip and later regret: without it, index drift is invisible until a user reports that something they know exists cannot be found.

Name the failure class properly. It is not “search is broken.” It is silent projection drift — the index and the source of truth diverge, no alarm fires, and the symptom surfaces as a business complaint rather than an engineering alert. The counter-measure is boring: a periodic count-and-checksum comparison per partition, with a threshold that pages someone.

Where vector search changes the picture — and where it doesn’t

Since 2023, every search engine in this space has grown an approximate-nearest-neighbour index. Elasticsearch and OpenSearch ship HNSW-based dense vector fields; Solr has dense vector search too. That has made “do I need a separate vector database?” one of the most common questions we get.

The honest answer is that it depends on how much of your retrieval quality comes from lexical matching. Hybrid retrieval — BM25 for keyword precision, dense vectors for semantic recall, fused with something like reciprocal rank fusion — consistently outperforms either alone on the mixed query distributions real users produce (observed-pattern, across retrieval-augmented generation systems we have tuned). If you need hybrid, an engine that already does both well is one fewer moving part.

Where a dedicated vector store earns its place is at high embedding dimensionality with heavy filtered-ANN traffic, or when the recall/latency curve needs tuning beyond what a general-purpose index exposes. We compare those trade-offs directly in our open-source vector database comparison for AI workloads, which covers index-type choice and filter-aware recall in more depth than this article does.

What has not changed: the vector index is still a projection. Re-embedding a corpus after a model upgrade is exactly the reindex problem again, with a GPU bill attached.

A five-question readiness check before you commit

Run this before signing off on a search-bearing architecture. Any “no” is a design gap, not a blocker — but it should be a recorded one.

1. Can you rebuild the entire index from a source you control, end to end, without the current cluster? If the runbook does not exist, write it before launch.
2. Do you know your index-to-source drift, numerically, right now? Not “we assume it’s fine.”
3. Is your relevance tuning reversible? Config in Git, not clicked into a UI.
4. Have you sized for the write amplification? Indexing throughput is bounded by analysis and segment merging, not raw disk — expect the index to cost meaningfully more per write than the source store (observed-pattern).
5. Does your licence and distribution choice survive the next procurement review? Elastic Licence, Apache 2.0 Solr, and OpenSearch’s Apache 2.0 fork have materially different implications for redistribution and managed-service resale.

Teams that can answer all five tend to run search infrastructure calmly. Teams that cannot usually discover which question they skipped during an incident.

FAQ

Is Elasticsearch a NoSQL database?

It is a distributed document store with a NoSQL data model, but its design centre is search, not storage. It stores JSON documents and can retrieve them by ID, yet its indexes are lossy, derived projections that usually require a full rebuild after mapping changes. Treat it as a search layer over a system of record you control, not as the record itself.

What is the practical difference between Elasticsearch and Solr?

Both sit on Apache Lucene, so core retrieval and BM25 relevance behave very similarly. The differences are operational and commercial: Solr is Apache 2.0 with explicit, version-controllable schema and ZooKeeper-based coordination, while Elasticsearch offers a larger API-first product ecosystem under the Elastic Licence — which is why the AWS OpenSearch fork exists. Licensing and ecosystem fit usually decide it before performance does.

When should I use a search engine instead of MongoDB or Cassandra?

Use a search engine when the query is relevance-shaped: full-text ranking, faceted navigation, fuzzy matching, or aggregations over text. Use a document or wide-column store when the query is identity-shaped — fetch by key, fetch by partition, write at high volume with transactional guarantees. Most real platforms run both and project from one into the other.

Do I still need a dedicated vector database if Elasticsearch supports kNN?

Not usually. If your retrieval quality benefits from hybrid lexical-plus-semantic ranking, an engine that already does both removes a moving part. A dedicated vector store earns its place at high dimensionality with heavy filtered ANN traffic, or when you need finer control over the recall/latency curve than a general-purpose index exposes.

How do I keep a search index consistent with the source of truth?

Combine change data capture for freshness with a scheduled full reindex for reconciliation, and monitor the divergence between them. Dual writes from the application are the common shortcut and the common cause of silent drift, because the two writes are not atomic and nothing reports the gap. A periodic per-partition count-and-checksum comparison with an alerting threshold is the minimum viable safeguard.

The question to settle before the shortlist

The useful decision is not “which NoSQL database.” It is: what is the authoritative copy of this data, and what is a projection of it? Once that line is drawn, Elasticsearch, OpenSearch, and Solr become straightforward choices about relevance, licence, and operational fit — questions with defensible answers.

Leave it undrawn and the shortlist argument will keep resurfacing, in a different vocabulary each time, until an incident settles it. If you are working through the storage-layer decisions on a cloud data platform, our cloud and data engineering work is where those boundary questions get resolved on paper before they get resolved in production.