This is the fourth day of my participation in the August More text Challenge. For details, see: August More Text Challenge
After the introduction and installation of Kafka in the last two articles, let me ‘write a HelloWorld’ using Kafka.
This time we will start the Kafka service through the command line, start the Kafka consumer service, start the Kafka producer service, send the message data through the producer service, and then achieve the message production – consumption ‘Helloworld’.
The following scripts are executed in the Kafka installation path /bin and tested by the author.
1. Start the ZooKeeper service
First let’s start the ZooKeeper service by specifying the configuration file zooKeeper.properties.
./zookeeper-server-start.sh -daemon .. /config/zookeeper.properties
Start the Kafka service
Run the following script to start the Kafka service and specify the configuration file server.properties
./kafka-server-start.sh .. /config/server.properties
Create a Topic
Create topic from a script.
./kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic kafka-topic-test1
Here are some important parameters to configure.
Replication-factor 1: The number of copies of a topic. In theory, there can be multiple copies of a topic and these copies will be evenly distributed among brokers. It is also important to note that the number of topic replicas cannot exceed the number of brokers.
— Partitions 1: Number of topic partitions, if there is only one broker, then by default they are all in that broker; If there are multiple brokers, Kafka provides a partitioning policy for a topic. In a distributed system, when consumers retrieve data, they can retrieve the data from which partitions the new data is available and then retrieve the relevant data from the corresponding partitions. There’s one thing that’s involved, and that’s offset, and we’ll come back to that in particular.
–topic kafka-topic-test1: Sets the topic name.
Query the list of existing topics
The query script is as follows:
./kafka-topics.sh --list --zookeeper localhost:2181
–list: Queries the topic list.
All topics can be obtained by querying.
Start the Kafka consumer service
Start executing the consumer service script
./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic kafka-topic-test1 --from-beginning
The related configuration parameters are also described here.
–from-beginning: Reading message data that has not been consumed.
Start the Kafka producer service
Starts executing the producer service script
./kafka-console-producer.sh --broker-list localhost:9092 --topic kafka-topic-test1
Let’s try the results
Let’s enter some text in the producer window, as shown in the figure below:
Then we go back to the consumer window and you see this:
So we have completed the basic operation of Kafka hello World, celebrate!
To summarize
And then we’ll talk a little bit more about the lifecycle of consumption data from the producer to the consumer.
If there is not observed, please point out in time, thank you.