Get to know Kafka for the first time and build quickly

Download and install

• A:

Download the TGZ package from the official website, upload it to the server and decompress it. The download address is kafka_2.12-2.1.0.tgz.

• Method 2:

Curl curl curl curl curl curl curl curl curl curl curl curl curl curl

Curl - 0 http://mirror.bit.edu.cn/apache/kafka/2.4.0/kafka_2.12-2.4.0.tgz tar - XZF kafka_2. 12-2.4.0. TGZcdKafka_2. 12-2.4.0Copy the code

nohup ./bin/zookeeper-server-start.sh config/zookeeper.properties &
Copy the code

nohup ./bin/kafka-server-start.sh config/server.properties &
Copy the code

Create a Topic

We can create a topic on the server side using the following command:

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic mytopic
Copy the code

• –create create command, and –delete delete topic
• –bootstrap-server Specifies the IP address of the connected node
• — Replication-factor topic number of copies
• — Number of partitions corresponding to the current topic. Partitions can be understood as queues.

View a list of topics that have been created:

bin/kafka-topics.sh --list --bootstrap-server localhost:9092
Copy the code

producers

Start a producer and enter interactive mode when the command is executed.

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic mytopic
> hello kafka!!!
> end.
Copy the code

• –broker-list List of nodes that can send messages to multiple hosts, HOST1:PORT1,HOST2:PORT2.

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic mytopic --from-beginning
> hello kafka!!!
> end.
Copy the code

Start a multi-node cluster

1. Create configuration files for each node (server.properties) :

cp config/server.properties config/cluster/server-00.properties
cp config/server.properties config/cluster/server-01.properties
cp config/server.properties config/cluster/server-02.properties
Copy the code

2. Modify the contents of each configuration file:

config/cluster/server-00.properties:
		broker.id=0
    listeners=PLAINTEXT://:9092
    log.dirs=/tmp/kafka-logs-00
    
config/cluster/server-01.properties:
		broker.id=1
    listeners=PLAINTEXT://:9093
    log.dirs=/tmp/kafka-logs-01
    
config/cluster/server-00.properties:
		broker.id=2
    listeners=PLAINTEXT://:9094
    log.dirs=/tmp/kafka-logs-02
Copy the code

3. Start Kafka Broker with the above configuration:

nohup ./bin/kafka-server-start.sh config/cluster/server-00.properties &
nohup ./bin/kafka-server-start.sh config/cluster/server-01.properties &
nohup ./bin/kafka-server-start.sh config/cluster/server-02.properties &
Copy the code

4. Cluster mode creates a topic:

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 1 --topic my-replicated-topic
Copy the code

5. Topic description:

bin/kafka-topics.sh --describe --bootstrap-server localhost:9092 --topic my-replicated-topic Topic:my-replicated-topic PartitionCount:1 ReplicationFactor:3 Configs: Topic: my-replicated- Topic Partition: 0 Leader: 1 Replicas: 1, 0 Isr: 1 0Copy the code

Result Meaning:

• Topic Indicates the current query Topic
• PartitionCount Specifies the number of partitions corresponding to the current theme
• Copies of ReplicationFactor Topic
• Leader Broker. Id of the Leader node
• Replicas node broker.id
• The replica node broker. Id of Isr that is currently active

List all topics

bin/kafka-topics.sh --zookeeper node01:2181 --list
Copy the code

bin/kafka-topics.sh --bootstrap-server broker_host:port --delete --topic <topic_name>
Copy the code