As a beginner Kafka, I need to grow up quickly and take on the responsibility of maintaining Kafka. I follow three steps to learn Kafka:
-
Read about Kafka
-
Learn Kafka from the perspective of operational combat
-
Read the source code, systematic, fine master its implementation principle
Kafka consumer groups operation and maintenance command Kafka consumer groups operation and maintenance command Kafka consumer groups operation and maintenance command
1. Kafka consumer group operation and maintenance combat
| options | type | Detailed instructions |
|---|---|---|
| –delete | The command | Delete the consumption group. After the consumption group is deleted, its corresponding consumption site will also be deleted. |
| This command cannot delete an online consumer group (with a client connection) | ||
| –list | The command | Query all consumer groups (list) |
| –describe | The command | View consumer group details |
| –reset-offsets | The command | Reset the consumer group site |
| –bootstrap-server | options | Specifies the address of the Kafka cluster |
| –members | options | Applies when –describe and is used to display client information for a consumer group without showing partition level |
| –state | options | Applies to –describe and is used to show the status of the consumer group, mainly including where |
| Coordinators, partition load algorithms, etc. | ||
| –timeout | options | Timeout duration, expressed in ms. The default value is 5000. |
| –topic | options | Specifies the topic, partition information for this operation, Kafka deletes a subscription, or resets a site |
| Support for resetting only partial partitions of partial topics in the following format: Topic: Partition list, example: | ||
| Topic1: “0”, | ||
| –verbose | options | Output some more detailed information, which can be used with –offsets –members –state |
| –all-topics | options | The reset locus (–reset-offsets) is applied to all topics subscribed to by the consumer group. |
| –by-duration | options | Reset point to displacement at a specified interval from the current time, format, PnDTnHnMnS, example |
| Reset 30 minutes before the format is PT0H30M0S | ||
| –to-current | options | Reset to the point represented by the current time |
| –to-latest | options | Reset to the maximum location |
| –to-datetime | options | Reset the minimum number of loci greater than the specified time in the format ‘YYYY-MM-DDTHh: MM: ss.sss’ |
| –to-offset | options | Reset to the specified location |
| –to-earliest | options | Reset to the minimum point |
| –shift-by | options | Resets the loci to the current shift + N, which can be negative to indicate forward movement |
| –dry-run | options | Just showing the results, not really executing, feels like simulating execution |
| –execute | options | Performs a true site reset. If this parameter is not taken, no site reset is actually performed |
| –export | options | Export the site reset plan to a file |
| –from-file | options | Specifies a site reset plan, which is generated by the –export parameter |
Next, make a simple explanation and example for some key points.
1.1 –members Displays the members of the consumer group
When viewing a consumption group using the –describe option, detailed consumption information for each partition is displayed, while –members is a group of client-side summary information, as shown in the figure below:
1.2 –state Displays the status of the consumer group
This option is used to display the status of the consumer group, as shown in the following figure:
The columns are described as follows:
-
COORDINATOR (ID) Control COORDINATOR of the consumer group, such as coordinating partition rebalancing operations. Different consumer groups have different brokers.
-
Assignments-strategy Partition load algorithm Kafka supports Rrange and RoundRobin allocation algorithms. The default Range algorithm is shown in the following figure.
Insert a picture description here
-
State Consumer states, including Dead, Stable(Stable, consumable messages), CompletingRebalance(in process of rebalancing), PreparingRebalance(ready to rebalance), empty(all consumption is not online)
-
MEMBERS Indicates the number of MEMBERS.
1.2 –reset-offsets reset consumption points
Resetting the consumption locus is a key feature of Kafka-consumer-groups. We demonstrate common functions that can only be remembered by doing them.
Kafka’s reset point requires that all consumers be stopped first.
The –execute option is required to perform the actual point reset, which can be simulated with –dry-run before the actual execution.
2, summarize
In Kafka, consumer groups are created automatically, and Kafka does not offer RocketMQ’s broadcast consumption model, nor does it have as many operational properties as a Topic. Understandably, a Topic is responsible for specific data storage, while a consumer group just pulls data and processes it.
Well, this article is introduced here, three even (attention, like, message) is the biggest encouragement to me, of course, you can add the author wechat: DINGwPMz, common exchange and discussion.
Walk into the author
Here’s some advice from a 10-year IT veteran for new employees
“I” was flattered by Ali Baba
How can programmers increase influence
How to read source code efficiently
Another way FOR me to get involved in the RocketMQ open source community