This is the 8th day of my participation in Gwen Challenge.More article challenges

The first developer, starting a producer, is opening a thread on the JVM. If you want to open multiple producers, each thread will be assigned a client ID: producer-1. Which components are then passed before sending messages to the broker.

An overview of the components,

Metadata -> RecordAccumulator -> Sender -> NetworkClient

  • Partitioner: The sent message is routed to the subscribed by certain partitioning rulestopiconepartition
  • metadataFrom:brokerThe cluster to obtaintopicMetadata ofTopic -> Partitions(Leader+Followers, ISR)), only then will you know what there arepartitionCan send. So whatever happens to your metadata, it’s going to affect this
  • RecordAccumulator: Message sending buffer. Messages sent to the specified partition are packaged into one by onebatch. For abrokerMultiple partitions ofbatchIt’s going to be packaged into onerequestTo send. Batch control parameters:batch.size(The tuning parameters are:this.accumulator = new RecordAccumulator()Parameters can be reflected in
  • Sender: Is responsible for retrieving messages from buffers and sending them to the broker
  • NetworkClient: the component of network communication, the underlying component that actually does the sending. Sealed inSenderinside
  • KafkaThread: is the thread that actually sends the data,name = kafka-producer-network-thread | clientId. inSenderWhen initialized, the thread is started.

Impact parameters

  1. metadata.max.age.ms: Sends a request to refreshbroker topicMetadata interval. Default value: 5 minutes
  2. max.request.size: Maximum number of bytes per request, 1M
  3. buffer.memory: Message buffer size, 32 MB
  4. retry.backoff.ms: retry interval, 100ms
  5. max.block.ms:send()Maximum time between blocks (cause: buffer blocking/metadata information unavailable). Default: 60s
  6. request.timeout.ms:senderSend a message tobrokerRequest timeout of The default 30 s
  7. batch.size: Assemble the record into batch as far as possible to reduce IO. The default 16 k
  8. linger.ms: when abatchThe size is not fully worked out and still requires a minimum delivery time. The default is 0, which means there is no time limit
  9. max.in.flight.requests.per.connection: conn on the same brokerrequestA maximum of five responses are not received
  10. acks: all indicates that the message is written successfully only when both the leader and follower receive the message. 1 indicates that the message is written by the leader. The default is 1
  11. retries: Number of retries. The default value is 0. If you need to retry, you can set it