❝

Message queues are also called Message Queues (MQ). Kafka is a high throughput distributed publish-subscribe messaging system. Kafka is a high throughput distributed publish-subscribe messaging system. Kafka is a high throughput distributed publish-subscribe messaging system.

❞

1. What is Kafka?

Apache Kafka is a publish-subscribe messaging system developed by Apache.

2. What are the three key features of Kafka?

  • Publish and subscribe record flows, similar to message queues or enterprise messaging systems.
  • Store record streams in a fault-tolerant persistent manner.
  • Process the record stream.

3. Kafka is generally used in two broad categories of applications?

  • Establish real-time streaming data pipelines to reliably retrieve data between systems or applications
  • Build real-time streaming applications to transform or respond to data streams

4. Kafka feature?

  1. Message persistence
  2. High throughput
  3. scalability
  4. Multi-client support
  5. Kafka Streams
  6. Security mechanism
  7. The data backup
  8. lightweight
  9. Message compression

Kafka’s five core apis?

  • Producer API

  • Consumer API

  • Streams API

  • Connector API

  • Admin API

6. What is a Broker?

In a Kafka cluster, a Kafka instance is called a Broker node.

What is a Producer?

The producers of messages are called producers.

Producer sends messages to be stored in a topic specified by the cluster, and also customizes algorithms to determine which partition to send message records to.

8. What is a Consumer?

The consumer of the message reads the message from the topic specified in the Kafka cluster.

9. What is Topic?

Topic. Kafka logs messages for different business types through different topics.

What is a Partition?

Each Topic can have one or more partitions.

11. Relationship between partition and proxy node?

A partition corresponds to only one Broker, and a Broker can manage multiple partitions.

12. What is Replication?

Each topic requires a number of copies when it is created (default: 1).

What is a Record?

Data that is actually written to a Kafka cluster and can be read by consumers.

Each record contains a key, value, and timestamp.

14. What scenarios does Kafka fit into?

Log collection, messaging systems, activity tracking, operational metrics, streaming, time sources, etc.

15. Kafka disk selection on?

It is well known that SSDS perform better than ordinary disks, which we actually use. It uses sequential read and write operations. To some extent, it avoids the biggest disadvantage of mechanical disks, that is, the slow random read and write operations. Therefore, SSD does not have much advantage.

16. Advantages of using RAID?

  • Provides redundant disk storage space
  • Provide load balancing

17. How many factors should be considered when planning disk capacity?

  • Number of new messages
  • Message retention time
  • Average message size
  • Number of backup
  • Whether to enable compression

18.Broker using a single? Multiple file directory path parameters?

More than the dirs

The dir individual

19. Generally, which parameter is selected to configure a path? Benefits?

log.dirs

Benefits:

Improved read/write performance: High read/write throughput on multiple physical disks at the same time.

Failover. One disk died and was transferred to another.

20. What are the parameters for automatically creating topics?

auto.create.topics.enable

21. Solve kafka message loss problem?

  • Instead of using producer.send(MSG), use producer.send(MSG, callback).
  • Set acks = all.
  • Sets retries to a larger value.
  • Set the unclean. Leader. Election. The enable = false.
  • Replication. factor >= 3.
  • Set min.insync.replicas > 1.
  • Make sure replica.factor > min.insync.replicas.
  • Ensure message consumption completes before submission.

22. How do I customize a zone policy?

Explicitly configure the producer-side parameter partitioner. Class

The parameter is the fully qualified class name of your implementation class.

23. Where can kafka compress messages occur?

Producers, the Broker.

Kafka message duplication problem?

Do idempotent.

The database aspect can avoid duplication (unique and primary keys).

Take control of the business.

25. What do you know about kafka monitoring tools?

  • JMXTool tools
  • Kafka Manager
  • Burrow
  • JMXTrans + InfluxDB + Grafana
  • Confluent Control Center

26. Kafka system supports two different sending modes.

Asynchronous mode

Synchronous mode

27. What are the differences between consumers and consumer groups?

A consumer group can have one or more consumer programs.

The consumer group name (GroupID) is typically represented by a unique string.

If a consumer group subscribes to a topic, each partition of that topic can only be assigned to one consumer program within one consumer group.

Reference:

  • Kafka is not hard to learn
  • Introduction to Kafka in Practice
  • Geek Time: Kafka core Technology and Practice
  • http://kafka.apache.org/

This article is formatted using MDNICE