Elasticsearch Fundamentals

Elasticsearch is a distributed search and analytics system built on Lucene. It is useful when you need fast full-text search, flexible filtering, and near real-time aggregations over large document collections. It is a poor fit if you treat it like a drop-in relational database, because its strengths and operational costs come from a very different storage model.

Start with the indexing pipeline

When you send a JSON document to Elasticsearch, the document is split into fields and each field is handled according to its mapping. Text fields are usually analysed. An analyser tokenises the text, normalises case, may fold accents, may remove stop words, and can stem words to a root form. The resulting terms are written into an inverted index, which maps each term to the documents that contain it.

That structure is what makes text search fast. A query for "distributed cache" does not scan every document. It looks up the posting lists for the relevant terms and combines them. Term positions can also be stored, which enables phrase queries, highlighting, and relevance scoring such as BM25.

Mappings are critical because the same logical field may need several representations. A product title might need a text field for full-text search and a keyword subfield for exact filtering and sorting. If you let dynamic mappings guess everything, you often end up with bloated indexes or the wrong query behaviour.

Understand shards, replicas, and refresh

An Elasticsearch index is divided into primary shards. Each shard is an independent Lucene index. Sharding allows parallel indexing and querying, but every shard has metadata, memory, and merge overhead. Too many small shards waste resources. Too few large shards make recovery and rebalancing painful.

Replicas copy primary shards to other nodes. They improve availability and increase read capacity, but they also increase storage use and cluster recovery time. Replicas do not make writes free. Every indexed document must still be replicated before the write is considered durable, depending on the request settings.

New documents are not instantly searchable. Elasticsearch periodically refreshes shards by opening recently written segments for search. A short refresh interval improves search freshness, but it creates more segments and more background merge work. Heavy indexing workloads often benefit from a longer refresh interval during bulk ingestion.

Querying is more than keyword matching

Elasticsearch combines full-text search, exact filters, and aggregations in one engine. Filters are usually cached and are ideal for structured constraints such as status, region, or date range. Full-text clauses contribute to scoring and are sensitive to the analyser used at index time and query time.

Aggregations make Elasticsearch useful for dashboards and exploration, but they can be memory-hungry, especially on high-cardinality fields. Field type choices matter here. keyword fields, doc values, and careful cardinality expectations often decide whether an analytical query is cheap or cluster-threatening.

Operational limits matter

Lucene segments are immutable, so updates are really delete-plus-add operations. Frequent updates on large documents can create heavy merge pressure. Heap sizing, disk throughput, and cluster state size all matter. So do index lifecycle rules, because old data left on hot nodes will eventually crowd out the workload that needs fast access.

Learning Elasticsearch therefore means learning workload shape. Choose mappings deliberately, keep shard counts boring, test analysers against real language data, and measure refresh, merge, and query costs under realistic load.

Elasticsearch fits logs, catalogues, document search, and operational analytics where flexible search matters. It fits poorly as a primary transactional store or where strict joins, cross-row transactions, and exact relational semantics are the main requirement.