How to setup zookeeper

How to setup zookeeper

What is zookeeper 

Zookeeper is an open-source project which provides distributed coordination services. It guarantees

  • Sequential consistency:  The order of the logs written will be the same as they were received.
  • Atomicity: No partial update will be done, either write operation will succeed or failed.
  • Single system images: It makes sure that all the zookeeper server have the same and exact data
  • Reliability: Once a log is written it will persist until the next update operation occurs.

These features make zookeeper ideal to use it for scenarios where you have to:

  • Cluster management
  • Leader election
  • Configuration management
  • Realtime node addition/deletion in cluster

How to setup Zookeeper

Prerequisites :

  1. java
  2. Monit/supervisor to ensure zookeeper is up and running

We will set up a zookeeper ensemble with 3 nodes with hostnames:

  • zk1
  • zk2
  • zk3

1. Download zookeeper stable version

wget https://downloads.apache.org/zookeeper/zookeeper-3.6.0/apache-zookeeper-3.6.0-bin.tar.gz

2. Unzip the tar file

sudo tar -zxvf apache-zookeeper-3.6.0-bin.tar.gz -C /opt && sudo mv /opt/apache-zookeeper-3.6.0-bin /opt/zookeeper

3. Create a directory for zookeeper data

mkdir -p /var/lib/zookeeper

5. Assign a unique id to each node

On node 1

echo 1 > /var/lib/zookeeper/myid

On node 2

echo 2 > /var/lib/zookeeper/myid

On node 3

echo 3 > /var/lib/zookeeper/myid

6. Edit the configuration file on each server on path /opt/zookeeper/conf/zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181
server.1=zk1:2888:3888
server.2=zk2:2888:3888
server.3=zk3:2888:3888

7. Start the zookeeper service

./zkServer.sh ../conf/zoo.cfg

Your zookeeper server must be up and running, happy bookkeeping!!

A few points for production-ready zookeeper :

  1. To handle node failures ensure you have 2* f+1 nodes in the ensemble, f being the failed nodes
  2. Ensure correct JVM size for proper functioning and throughput
  3. Enable authentication
  4. Never opt for default configuration provided in zookeeper binary

 

Zookeeper monitoring/troubleshooting will be covered in the next blog. 

Leave a Reply

Your email address will not be published. Required fields are marked *