Install Kafka on the Linux server

  • The basic concept
  • Download the installation package
  • Creating an installation directory
  • Decompress the installation package
  • Configuration kafka
  • Start the kafka
  • Verify the kafka
    • Validate topic
    • Verify producer producer
    • Verify consumer
  • Close the kafka

The basic concept

  • Before installing Kafka, make sure you have the JDK and ZooKeeper installed in your server environment

Download the installation package

  • Download the kafka server
  • Download using the wget command:
Wget HTTP: / / https://downloads.apache.org/kafka/2.6.0/kafka_2.13-2.6.0.tgzCopy the code

Creating an installation directory

  • Create kafka installation directory:
cd DATA
mkdir kafka
Copy the code

Decompress the installation package

  • Save the downloaded Kafka installation package to the kafka directory you created
  • Unzip kafka installation package to kafka directory:
The tar - ZXVF kafka_2. 13 - server. TGZCopy the code

Configuration kafka

  • Go to the kafka directory
  • Go to the Kafka config directory
  • Configure the kafka server.properties configuration file
CD kafka_2.13-2.6.0 CD config vi server.propertiesCopy the code
broker.id=0
# port
port=9092
# host address: 127.0.0.1
host.name=127.0.0.1
Path for storing logs
log.dirs=/ DATA/kafka kafka_2. 13 - server/log
127.0.0.1 is the address and port of the zooKeeper server
zookeeper.connect=127.0.0.1:2181
Copy the code
  • Configure the zookeeper.properties configuration file for Kafka
vi zookeeper.properties
Copy the code
# ZooKeeper data directory
dataDir=/ DATA/kafka kafka_2. 13 - server/zookeeper/DATA/dataDir
# ZooKeeper log directory
dataLogDir=/ DATA/kafka kafka_2. 13 - server/zookeeper/DATA/dataLogDir
clientPort=2181
maxClientCnxns=100
tickTime=2000
initLimit=10
Copy the code

Start the kafka

  • Start ZooKeeper before starting Kafka. You can create batch startup scripts:
vi startKafka.sh
Copy the code
#Start the zookeeper/ DATA/kafka kafka_2. 13 - server/bin/zookeeper - server - start. Sh/DATA/kafka kafka_2. 13 - server/config/zookeeper. The properties &#The default wait time is 2 seconds
sleep 2
#Start the kafka/ DATA/kafka kafka_2. 13 - server/bin/kafka - server - start. Sh/DATA/kafka kafka_2. 13 - server/config/server properties &Copy the code
#Example Add script execution permission
chmod +x startKafka.sh
Copy the code
  • Start the zookeeper:
zkserver.sh start
Copy the code
  • Execute the script to start kafka:
startKafka.sh
Copy the code

Verify the kafka

Validate topic

  • Create topic in kafka bin
Kafka-topics. Sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partion 1 --topic nameCopy the code
  • Kafka bin = topic; kafka bin = topic;
Kafka - switchable viewer. Sh - the list - the zookeeper 127.0.0.1:2181Copy the code
  • Delete topic from kafka bin
Kafka-topics. Sh --delete --zookeeper 127.0.0.1:2181 --topic nameCopy the code

Verify producer producer

  • In kafka’s bin directory, execute the command to start producer producer:
Kafka-console-producer. sh --broker-list 127.0.0.1:9092 --topic nameCopy the code

Verify consumer

  • Start consumer in kafka bin:
Kafka-console-consumer. sh --bootstrap-server 127.0.0.1:9092 -- Topic name --from-beginningCopy the code

Close the kafka

  • You can create a shutdown kafka script:
vi stopKafka.sh
Copy the code
#Close the kafka/ DATA/kafka kafka_2. 13 - server/bin/kafka - server - stop. Sh/DATA/kafka kafka_2. 13 - server/config/server properties &#Close the zookeeper/ DATA/kafka kafka_2. 13 - server/bin/zookeeper - server - stop. Sh/DATA/kafka kafka_2. 13 - server/config/zookeeper. The properties &Copy the code
  • Execute the script to disable kafka:
stopKafka.sh
Copy the code
  • You must disable Kafka first and then ZooKeeper. Otherwise, Kafka will continue to connect to ZooKeeper in an endless loop
  • If Kafka enters an infinite loop and cannot be shut down, the Kafka server process must be forcibly killed