How to provision production grade Elasticsearch

How to provision production grade Elasticsearch

There is lots of confusion on how to set up a production-grade elasticsearch cluster. Although there is no silver bullet but you can have a set of standard parameters which you can follow to ensure smooth functioning of your cluster. Below is the gist of such configuration :

  1. Disable swap entirely

    The swap memory is a provision where the system keeps working even when enough memory is not available by using the hard disk as a memory at a cost of reduced performance. With setting, elasticsearch will become slow due to the high I/O cost involved with the disk.

  2. Increase file descriptor

    Since everything in Linux is a file make sure you allow many open file descriptors as you might require for elasticsearch threads.

  3. Have at least 4096 threads

    As a thumb rule allow at least 4096 threads, this is the minimum number suggested.

  4. Tune JVM DNS cache settings

    In most cases, you might not require this config but this a good thing to have. In any environment where DNS plays a key role a proper TTL can be a life savior. Suppose your master with DNS es1.yourdomain.local with IP 1.1.1.1 has a new IP address assigned 192.1.1.1 but due to improper TTL cache your other nodes still try to access the old IP address. This could lead to downtime so tune the JVM DNS cache.

  5. Enable mlockall

    mlockall ensures elasticsearch blocks the desired memory on startup for smooth functioning. So this will make sure it does not starve for the memory.

  6. Have a minimum of 3 nodes

    Whenever using elasticsearch for production make sure to have a 3 node cluster. It will give you some breather in case there is any glitch with your nodes. Also, it will be easy for you to do operational stuff like upgrade, etc without downtime.

  7. Provision nodes with dedicated roles

    Having dedicated client, data and master nodes on separate boxes takes your cluster to the next level of availability and fault tolerance. Only one thing you need to take cares is the cost implications.

Leave a Reply

Your email address will not be published. Required fields are marked *