1. Primary Shard and Replica Shards
Elasticsearch distributes data across multiple shards, which can be located on different servers (nodes) within the cluster. Each shard includes one primary shard and multiple replica shards. The primary shard handles write operations and some read operations, while replica shards primarily manage read operations and serve as backups for the primary shard in case it fails.
2. Shard Allocation
When a document is indexed in Elasticsearch, it is first written to the primary shard. Subsequently, the document is asynchronously replicated to all configured replica shards. Elasticsearch's cluster management component automatically handles shard allocation across nodes and reassigns shards as needed to maintain cluster balance.
3. Fault Tolerance
If the node hosting the primary shard fails, Elasticsearch selects a new primary shard from the replica shards. This ensures service continuity and data availability. The system continues to process write operations via the new primary shard and can also handle read operations.
4. Data Synchronization
Replica shards periodically synchronize data from the primary shard. This means that even during hardware failures or network issues, all data changes are preserved and can be recovered from replica shards.
Example
Suppose an Elasticsearch cluster has 3 nodes, with an index configured for 1 primary shard and 2 replicas. When a document is written to the index, it is first stored on the primary shard and then replicated to the two replica shards. If the node hosting the primary shard fails, the cluster automatically selects a replica shard as the new primary shard and continues to serve. This ensures data is not lost and indexing operations can continue even if the original primary shard is unavailable.
Through this approach, Elasticsearch ensures data persistence and reliability while providing high-performance read and write capabilities. This high level of data replication and fault tolerance makes Elasticsearch well-suited for large-scale applications requiring high availability and fault tolerance.