[go] www.cnblogs.com/caoweixiong…

Business needs in the configuration of a Kafka cluster use, has been sa configuration, this time due to the more urgent, so according to the above big man article configuration, in this record memo. Here again, prepare three machines to install the Kafka cluster

Installed a.

The downloaded ZooKeeper version depends on JDK1.8 or later. An error message is displayed if the version is incorrect

  1. download

    Wget HTTP: / / https://www.apache.org/dyn/closer.cgi?path=/kafka/2.7.0/kafka_2.12-2.7.0.tgz

  2. Decompress tar XZVF kafka_2.12-2.1.0.tgz

Configuration of two.

  1. Creating a file Directory
Mkdir /data/kafka_2.12-2.7.0/zookeeper mkdir /data/kafka_2.12-2.7.0/log Mkdir /data/kafka_2.12-2.7.0/log/kafkaCopy the code
  1. Zookeeper configuration

Each machine editing files almost similar, vim/kafka_2 12-2.7.0 / config/zookeeper. Properties, are as follows

DataDir =/data/kafka_2.12-2.7.0/ zooKeeper dataDir=/data/kafka_2.12-2.7.0/ zooKeeper dataDir=/data/kafka_2.12-2.7.0/ zooKeeper DataLogDir =/data/kafka_2.12-2.7.0/log/zookeeper Add tickTime=2000# as zK basic time unit, InitLimit =10 # leader-follower initial communication duration tickTime*10 syncLimit=5# leader-follower synchronous communication duration tickTime*5 # Set the broker 0=10.5.10.1:2888:3888 server.1=10.5.10.2:2888:3888 server.2=10.5.10.3:2888:3888Copy the code
  1. Added the myID configuration for the ZooKeeper data directory

Generate file ids from the data directory on each machine based on the server that broker ids correspond to in the configuration file above

CD /data/kafka_2.12-2.7.0/zookeeper echo 1 > myid # echo 2 > myidCopy the code
  1. Kafka configuration

12-2.7.0 / config/vim/kafka_2 server properties, major changes broker. Id, advertised. Listeners, num. Partitions (here is the three machines), the dirs

# # # # # # # # # # # # # # # # # # # # # # # # # # # # # Server Basics # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # broker globally unique code, Can't repeat For example here is 0, the broker. Id = 1 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Socket Server Settings # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # IP configuration to monitor, modify for the machine advertised. Listeners = PLAINTEXT: / / 172.16.10.91:9092 # processing network request the number of threads, the default num.net work. The threads = 3 # to deal with disk IO number of threads, Bytes =102400 # Buffer size for receiving sockets. Default socket, the receive buffer. The bytes = 102400 # request socket buffer size, The default socket. Request. Max. 104857600 bytes = # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Zookeeper # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Configuration of three service zookeeper connection address zookeeper. Connect = 10.5.10.1:2181,10.5. 10.2:2181,10.5. 10.3:2181 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Log Basics # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # kafka run log repository paths the dirs = / data/kafka_2. # 12-2.7.0 / log/kafka topic in the current broker subdivision number, Num. Partitions =3 # Number of threads used to restore and clean data from data, The default num. Recovery. Threads. Per. Data. The dir = 1 # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Log Retention Policy # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Hours =168; log.extension. hours=168Copy the code

Three.

CD / / XXX kafka_2. 12-2.7.0

When Kafka starts, start ZooKeeper first and then Kafka. On the contrary, disable Kafka and then ZooKeeper

  1. Start the zookeeper

Start ZooKeeper on the three machines respectively

Bin/zookeeper server - start. Sh config/zookeeper properties & # background start nohup bin/zookeeper - server - start. Sh Config/zookeeper. Properties > / data1 / kafka_2. 12-2.7.0 / log/zookeeper/zookeeper. Log > 1 & 2Copy the code
  1. Start the kafka

Start Kafka on three machines separately

Nohup bin/kafka-server-start.sh config/server.properties & # nohup bin/kafka-server-start.sh config/server.properties 12 - > / data1 / kafka_2 2.7.0 / log/kafka/kafka. Log > 1 & 2 ` `Copy the code